An observation schema for public AI answers

Identity and time

Give every scheduled observation a stable identifier before the result is known. Use an unambiguous timestamp with an explicit offset, such as an RFC 3339 representation. Record the collection window separately when timing is grouped for analysis.

Identifiers should survive correction and publication. A corrected record becomes a version or successor; it should not silently replace the identity of the original observation.

Surface and question context

Record provider, public product surface, access route, visible version label when available, locale, session rule and configuration relevant to the protocol. Store a stable question ID and the exact submitted text when permitted.

Do not add fields for private ranking score, hidden query or chain of thought simply because they would be convenient. A schema should model evidence the study can actually obtain.

Outputs, sources and transformations

Keep the visible source URL apart from normalized URL, registered domain and analyst-assigned source class. Record the transformation code version. If answer text cannot be retained, store only the permitted derivative and name its limitation.

Inference belongs in a separate analysis table or clearly typed fields. This preserves the difference between what the surface returned and what the researcher concluded.

• Raw captured value, when permitted.
• Normalized value and transformation version.
• Missing reason and eligibility state.
• Rights or redistribution state.
• Protocol, collector and correction versions.

Validate before accepting a row

Use machine validation for required types, controlled vocabularies, timestamps, stable identifiers and referential links. Add study-specific checks for surface, locale and protocol compatibility.

Validation cannot prove that a capture is truthful, but it can prevent structurally ambiguous records from entering the release. Preserve rejected rows and reasons in an audit log.