AF-EX-2026-08-22 · method 0.1 · published 22 August 2026

A result you can reproduce—not a benchmark.

This worked protocol uses only owner-created synthetic answers and reserved .example source hosts. It demonstrates the Evidence Lab calculation contract. It does not compare real providers, measure model quality or support a market claim.

Download exact JSON Open the Evidence Lab

Accountability

Research owner: Matthias Ramahi
Protocol version: Method 0.1; assigned and maintained by Matthias Ramahi
Corrections: Affected record, reason, date, reviewer status and comparability effect are versioned by the Research Owner.
Reviewer status: No independent reviewer assigned; independent review is not claimed.
Provider/API cost: EUR 0; this example makes no provider or API call.

Reproduction steps

01
Open Evidence Lab and choose Load synthetic demo.
02
Confirm the exact question, four criteria and three observations shown below.
03
Choose Analyze observations. The deterministic method normalizes visible hosts, performs literal case-insensitive phrase checks and computes mean pairwise token Jaccard after the published stop-word filter.
04
Compare the five summary values with the dated JSON fixture. The generated timestamp may differ; inputs and calculated values must not.

Expected result

Observations: 3
Unique domains: 3
Recurring domains: 2 / 3
Answer overlap: 62%
Literal coverage: 12 / 12

Exact inputs

Question

What should a small team record before comparing public AI answers over time?

Criteria

observation timestamp
public surface
visible source URLs
missing data rule

Demo A2026-08-01T09:00 · en-US

Surface: Synthetic surface A
Version: Synthetic version A
Route: Synthetic route A

Record an observation timestamp, the public surface, locale and visible source URLs. Keep a missing data rule beside the record.

Visible hosts · method.example · schema.example

Demo B2026-08-08T09:00 · en-US

Surface: Synthetic surface B
Version: Synthetic version B
Route: Synthetic route B

A repeatable record needs an observation timestamp, public surface, question and visible source URLs. State a missing data rule before comparison.

Visible hosts · method.example · rights.example

Demo C2026-08-15T09:00 · en-US

Surface: Synthetic surface C
Version: Synthetic version C
Route: Synthetic route C

Record the public surface, locale, question, observation timestamp and any missing data rule. Preserve visible source URLs when retention is allowed.

Visible hosts · method.example · schema.example

What this proves—and what it cannot

It proves: with these disclosed inputs and method 0.1, the local tool produces the values above. The fixture creates an inspectable regression target for future corrections.

It does not prove: anything about ChatGPT, Gemini, Perplexity or another provider. No provider output was queried or retained, the example is not independently reviewed, and recurring hosts do not establish source quality or answer accuracy.

Future real-output research still requires a frozen protocol, independent reviewer status, a cost ceiling, provider-specific retention and publication review, and comparable batches.