3.4 Authenticity

A reseller can sell a frontier model and serve a cheaper one, count a cached prefix as fresh input, or answer in an API it only pretends to speak. None of that shows up in the traffic.

Authenticity asks the upstream directly, and comes back with a score out of 100 and a grade from A to F.

Nothing to pressCopy Anchor Link

Every provider is probed on its own:

• when it is added,
• when its endpoint, type or key changes,
• when it has never been probed,
• and again whenever its report goes stale.

The grade is on the Authenticity page and above the agents on the home page.

A probe spends a few cents of that provider's own credit, which the report states next to the finding.

What the test cases askCopy Anchor Link

The score is only a summary of the cases behind it, and every one of them is on the Test cases tab, naming the question it puts to the upstream, the exact request it sends, how the answer is judged, and what it is worth.

Half of them ask what the upstream is:

• Is the model that answers the one that was asked for — worth half credit off a vendor's own endpoint, where that field is whatever the upstream typed there.
• Which vendor does the model say trained it.
• Was anything injected in front of the request.
• Do several identical requests come back from the same model at all.
• Is a documented parameter (logprobs, n, a stop sequence) honoured, refused, or accepted with a 200 and quietly dropped.
• A test bank of questions with one right answer, from counting the letters in a word to who wrote the Preface to the Pavilion of Prince Teng.

The rest read the envelope:

• Does a two-level tool schema survive a forced call.
• Does the event stream carry everything the API documents.
• Is the prompt cache really accounted for.
• Are two identical requests billed the same.
• Are the vendor's own headers there.

Make it your ownCopy Anchor Link

Reweight a case, turn it off, rewrite its question, or add one of your own. The questions worth asking of a reseller are not the same everywhere, and a score whose method is not published is not evidence.

Restore defaults puts the shipped suite back and leaves your own cases alone.

The half that costs nothingCopy Anchor Link

The report has a second half that sends no request at all: what the records Gateway already kept say about that upstream —

• how much of the cache it really accounted for,
• how many attempts failed,
• how long it took to answer,
• and how much of what it served has no price.

Next: 3.5 Import settings and links.