]> Falsify OÜ

Estonia cuneyt@falsify.dev

Security SCITT transparency evaluation provenance This document defines a profile for carrying pre-run evaluation criteria as a SCITT Signed Statement payload, using the architecture of RFC 9943. It specifies the payload media type, the selection of the Issuer and Subject CWT claims, the encoding of hash-only statements for criteria that must remain confidential, a sequencing requirement that makes amendment order verifiable, and the semantics of amendment itself. It does not define a new transparency architecture; it describes how an existing artefact type is carried by the one RFC 9943 already defines.

Introduction An evaluation of an AI system is reported as a claim: a metric, a comparator, a threshold, a dataset and a seed. A reader of such a report cannot tell from its prose when that combination came into existence. A threshold chosen after the numbers were seen and a threshold chosen beforehand are textually identical. Where the evaluation data is withheld, which condition 4.1 explicitly permits in order to protect evaluation integrity, the reader additionally cannot tell whether an artefact disclosed later is the same artefact that existed at the time. already provides the machinery to evidence both points: a Signed Statement registered with a Transparency Service, optionally made over the hash of a payload rather than its bytes (, Section 6.2), and a defined way to supersede a statement when the state it describes changes (, Section 6.3). What is missing is a payload profile: an agreement on what the payload is, what identifies it across revisions, and how a verifier canonicalises it before comparing digests. This document supplies that profile for PRML , an existing serialisation of pre-run evaluation criteria with a registered media type.

Scope and Non-Goals This profile is deliberately narrow. What a conforming Statement establishes depends on the mode, and the difference matters. In full-payload mode, the canonical bytes of a specific criteria object were signed and registered no later than the time recorded by the Transparency Service. In hash-only mode, a commitment to a digest was signed and registered no later than that time. A Relying Party holding only the ledger entry cannot know that any object with that digest exists: a digest is 32 octets and committing to one is possible without a preimage. What the commitment gives is binding after the fact -- a later-revealed object that canonicalises to the committed digest is cryptographically bound to that commitment, and one that does not is not the committed object. Implementations MUST NOT describe hash-only registration as establishing the existence of the object before it is revealed. Neither mode establishes: that the evaluation run occurred after registration. Execution linkage is out of scope; a separate dated artefact is required and is not defined here. that the criteria are appropriate for the system under evaluation. anything about the correctness of the reported result. any property of the Issuer's independence. treats assessment carried out in terms set by the party being evaluated as a lower standard of independence than an independent audit; registration of a criteria object does not change that classification. Implementers and specification readers are asked to preserve these boundaries when summarising this profile.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Criteria object:

A PRML manifest, as defined in , carrying at minimum the nine required fields version, claim_id, created_at, metric, comparator, threshold, dataset, seed and producer.

Canonical bytes:

The byte string produced by the PRML canonicalisation algorithm ().

Payload Type The payload type (COSE header parameter 3, content type) of a conforming Signed Statement MUST be application/vnd.prml+yaml. This media type is registered in the IANA Media Types registry. Registration records a media type; it does not constitute review or endorsement of this profile by IANA.

Canonicalisation A digest over a serialisation format with degrees of freedom is ambiguous. Before hashing, a criteria object MUST be rendered to canonical bytes. PRML canonicalisation is defined as the output of a YAML block-style serialisation with keys sorted lexicographically, no flow style, Unicode permitted unescaped, and an effectively unbounded line width. The normative definition and its conformance vectors are in . The digest algorithm for this profile is SHA-256. Implementations MUST NOT hash the bytes as received from a producer. Two producers expressing identical criteria will generally not emit identical bytes; only the canonicalised form is comparable.

CWT Claims , Section 6 requires the protected header of a Signed Statement to include the CWT Claims header parameter carrying at least the Issuer claim (label 1) and the Subject claim (label 2). This profile constrains their values.

Issuer The iss claim identifies the Issuer that signs the criteria Statement, cryptographically bound as required by , Section 5.1.1.1. The Issuer is not necessarily the party that submits the Statement. allows a Client to register Signed Statements on behalf of one or more Issuers, so a conforming deployment may have an evaluation platform register on an evaluator's behalf. This profile constrains who signs, not who submits. A criteria object also carries a producer field. That field is self-asserted text inside the payload and is NOT authenticated by registration. Where the two disagree, the iss claim is authoritative as to who signed the Statement, and the producer field is to be read only as the Issuer's own description of the originating party. Verifiers MUST NOT treat the producer field as an authenticated identity.

Subject The sub claim MUST be the claim_id of the criteria object. This choice is not an invention of this profile. , Section 6.3 says that an Issuer aware of a changed state SHOULD register a new Signed Statement using the same iss and sub claims. , Section 6.1 independently requires that a producer changing any field of a previously registered claim create a new manifest whose prior_hash is the manifest hash of the previous claim, and that the new manifest "MUST retain the claim_id of the previous claim". claim_id is therefore already normatively stable across an amendment chain, while the payload digest changes with each revision. If sub were instead the digest of the criteria object, every amendment would carry a different sub and the relationship between successive versions would be lost. Binding sub to claim_id yields a stable subject and a changing body, which is what , Section 6.3 assumes.

Unprotected Header , Section 6.3 requires that the unprotected header of a Signed Statement be set to an empty map before the Statement can be included in a Statement Sequence. A conforming Statement under this profile therefore carries an empty unprotected header, and this profile defines no parameters to place there.

Detached (Hash-Only) Statements Where the criteria object must remain confidential, the Statement MUST be made over the SHA-256 digest of the canonical bytes rather than over the bytes themselves, as permitted by , Section 6.2 for payloads that are too large or too sensitive to transmit. This is the mode intended for the case in condition 4.1, where an evaluator withholds testing data. The dataset and thresholds are not transmitted to the Transparency Service and do not appear on the ledger.

Encoding , Section 6.2 permits a Statement over a payload hash but does not specify how that hash is encoded. Interoperability requires an exact answer, so this profile fixes one. In hash-only mode: The Statement payload MUST be the 32 raw octets of SHA-256 over the canonical bytes (), carried as a CBOR byte string. It MUST NOT be hex-encoded, base64-encoded, or wrapped in any further structure. The payload is present in the COSE_Sign1 rather than detached, so that a Relying Party holding only the ledger entry can read the committed digest without holding the criteria object. Publishing the digest is the point of this mode; a detached payload would put nothing on the ledger to compare a later disclosure against. iss, sub and the amendment semantics of are unchanged.

Mode Signalling -- Open Issue A verifier must be able to distinguish the two modes before parsing. Thirty-two octets of digest and a YAML manifest are both legitimate payloads under this profile, and a verifier that reads the first as the second fails in a confusing way rather than a safe one. Full-payload mode uses the payload type application/vnd.prml+yaml (). Hash-only mode requires a distinct signal, and this document does not fix one. The two candidates are a separately registered payload type for the digest form, or a COSE header parameter declaring the digest algorithm over the committed object. Until it is settled, an Issuer MUST NOT assert application/vnd.prml+yaml as the payload type of a hash-only Statement: 32 octets of digest are not a YAML manifest, and a false type is worse than an absent one. An Issuer SHOULD omit the payload type in this mode and convey it out of band. This is an interim measure, not the intended end state. This is the one open interoperability question in this profile. It is left open deliberately rather than settled unilaterally in a -00, and comment is invited.

Reveal A later disclosure to a party entitled to receive it is verified by canonicalising the disclosed object and comparing its digest to the registered one. A disclosure that does not reproduce the digest is not the registered object. Nothing in this mechanism obliges an Issuer to disclose, or evidences that a disclosure is complete.

Multiple Registrations An Issuer may register several criteria objects under the same sub and disclose only the one that suits a later narrative. This profile does not prevent that, and implementers MUST NOT describe it as preventing it. A Relying Party concerned with selective disclosure SHOULD use a Transparency Service or Auditor interface capable of enumerating all Statements for the relevant iss/sub pair. The append-only Statement Sequence (, Section 5.1.3) is what makes such completeness auditable; this profile does not define the discovery API, and does not require a Transparency Service to offer subject-based query.

Amendment Semantics A threshold that changes after testing is not, by itself, evidence of bad faith. Public evaluation guidance recognises legitimate post-hoc refinement with justification. What is objectionable is an undated change presented as the original. Accordingly, this profile does not treat a criteria object as immutable. It treats the history as immutable. An Issuer amending criteria MUST register a new Signed Statement under the same iss and sub per , Section 6.3, rather than attempting to replace an existing registration, and the new payload MUST set prior_hash to the digest of the superseded object per , Section 6.1. The earlier statement remains on the ledger. , Section 6.2 requires that verifiers treat the latest manifest in a chain as operative while also exposing the full chain on request; this profile adopts that requirement unchanged. The sequence of statements under a sub is the change history, and a justified later change is dated rather than hidden.

Two Complementary Histories A conforming deployment carries two records of the same sequence. They are not two independent time sources, and must not be presented as such. The prior_hash chain is intrinsic and Issuer-asserted. It is a link structure written into the payload by the party making the claim; it establishes which object supersedes which, and nothing about time. The created_at field is likewise Issuer-asserted. The ledger is extrinsic and Service-asserted. records the registration time as the time the Transparency Service added the Signed Statement to its Verifiable Data Structure, which the Issuer does not control. The ledger does not authenticate created_at. A verifier MUST treat created_at as Issuer-asserted, and the registration time as the independently evidenced upper bound on the object's existence. In particular, a created_at earlier than the registration time is consistent with the record and is not evidence of anything: an object may genuinely have been authored well before it was registered. Back-dating created_at is therefore NOT detectable from the ledger, and implementers MUST NOT claim otherwise. What checking both records does detect: a created_at later than the Statement's own registration time, which is self-contradictory; amendments registered in an order that disagrees with the prior_hash links; a Transparency Service that omits or reorders entries, which the prior_hash links contradict. Verifiers SHOULD check both and MUST report a disagreement between them rather than selecting one.

Sequencing Requirement The ledger order is not, by itself, evidence of issuance order. , Section 9.1 is explicit: "Unless advertised in the TS Registration Policy, the Relying Party cannot assume that the ordering of Signed Statements in the VDS matches the ordering of their issuance." A comparison between the prior_hash chain and the ledger is therefore meaningless unless something makes the two comparable. This profile supplies that something as its own requirement rather than assuming it of the Transparency Service:

• An Issuer MUST obtain a Receipt for a predecessor Statement before registering a successor that names it in prior_hash.

For a conforming Issuer the registration order of a chain is then necessarily the amendment order, and a ledger that shows otherwise indicates either a non-conforming Issuer or a Transparency Service fault. Verifiers MUST NOT apply the check in step 6 to chains whose Issuer does not claim conformance with this requirement.

Verification Procedure Steps 1 and 3 through 6 are common to both modes. Step 2 and step 7 differ, because the two modes register different things. Given a criteria object and a Receipt, a verifier: verifies the Receipt as a COSE Receipt per ; determines the mode: Full-payload mode: confirms the payload type is application/vnd.prml+yaml () and that the payload is the canonical bytes. Hash-only mode: confirms the payload is 32 octets (). How the two modes are distinguished on the wire is unresolved in this revision; until it is settled, a verifier obtains the mode out of band, and an implementation MUST NOT infer full-payload mode from the payload type alone when a 32-octet payload is possible. confirms the sub claim equals the claim_id of the object under examination; canonicalises the object per and computes its SHA-256 digest; compares that digest to the registered payload (full-payload mode) or to the committed digest (hash-only mode); where the object carries prior_hash, and only where the Issuer claims conformance with , resolves the chain and checks that its order is consistent with the ledger order of the corresponding Statements; reports the Transparency Service's recorded time as an upper bound -- in full-payload mode, on the existence of the object; in hash-only mode, on the registration of the commitment. The object's own existence is evidenced only once a disclosure reproduces the digest. A verifier MUST report step 7 as an upper bound. It is not evidence of when the evaluation ran.

Security Considerations The guarantees of this profile are those of the underlying Transparency Service; it adds no cryptographic mechanism of its own. Canonicalisation is security-relevant. An implementation that hashes non-canonical bytes will produce digests that fail to match for semantically identical objects, and may be induced to accept a non-matching object as matching if it normalises after comparison rather than before. Implementations SHOULD validate against the published conformance vectors. Registration is not authentication of content. A Transparency Service authenticates the Issuer of a statement, not the truthfulness of the payload. A registered criteria object is evidence of existence and integrity, not of accuracy. Hash-only mode publishes the digest and the time. It is a commitment, not a concealment scheme: this profile provides no hiding against a dictionary attack over a low-entropy criteria object. Where the plausible criteria space is small enough to enumerate -- a handful of metrics against round-numbered thresholds, for instance -- an adversary can recover the committed object by trying candidates until one reproduces the digest. defines no commitment nonce, and claim_id cannot serve as one because this profile publishes it as sub. Issuers for whom this matters should not rely on hash-only mode for confidentiality. Adding a salt would change the PRML object model and is not proposed here.

IANA Considerations This document has no IANA actions. The media type application/vnd.prml+yaml is already registered.

Traceability in supply chains is a growing security concern. While Verifiable Data Structures (VDSs) have addressed specific issues, such as equivocation over digital certificates, they lack a universal architecture for all supply chains. This document defines such an architecture for single-issuer signed statement transparency. It ensures extensibility and interoperability between different transparency services as well as compliance with various auditing procedures and regulatory requirements. CBOR Object Signing and Encryption (COSE) Receipts prove properties of a Verifiable Data Structure (VDS) to a verifier. VDSs and associated Proof Types enable security properties, such as minimal disclosure, transparency, and non-equivocation. Transparency helps maintain trust over time and has been applied to certificates, end-to-end encrypted messaging systems, and supply chain security. This specification enables concise transparency-oriented systems by building on Concise Binary Object Representation (CBOR) and COSE. The extensibility of the approach is demonstrated by providing CBOR encodings for Merkle inclusion and consistency proofs. This document describes how to include CBOR Web Token (CWT) claims in the header parameters of any CBOR Object Signing and Encryption (COSE) structure. This functionality helps to facilitate applications that wish to make use of CWT claims in encrypted COSE structures and/or COSE structures featuring detached signatures, while having some of those claims be available before decryption and/or without inspecting the detached payload. Another use case is using CWT claims with payloads that are not CWT Claims Sets, including payloads that are not CBOR at all. Concise Binary Object Representation (CBOR) is a data format designed for small code size and small message size. There is a need to be able to define basic security services for this data format. This document defines the CBOR Object Signing and Encryption (COSE) protocol. This specification describes how to create and process signatures, message authentication codes, and encryption using CBOR for serialization. This specification additionally describes how to represent cryptographic keys using CBOR. This document, along with RFC 9053, obsoletes RFC 8152. Concise Binary Object Representation (CBOR) is a data format designed for small code size and small message size. CBOR Object Signing and Encryption (COSE) defines a set of security services for CBOR. This document defines a countersignature algorithm along with the needed header parameters and CBOR tags for COSE. This document updates RFC 9052. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes the format of a request sent to a Time Stamping Authority (TSA) and of the response that is returned. It also establishes several security-relevant requirements for TSA operation, with regards to processing requests to generate responses. [STANDARDS-TRACK] AI Evaluator Forum Falsify OU

Relationship to Other Work Several individual Internet-Drafts published in 2026 define SCITT profiles for AI-related artefacts, including agent action authorisation and transparency receipts for regulatory disclosure obligations. This profile does not claim priority over or compatibility with any of them. It addresses a different artefact: the evaluation criteria themselves, before a run.

Worked Examples Two Signed Statements over the same criteria object, one per mode, are published alongside this draft together with the public key, the canonical bytes, the generator and an independent verifier that follows step by step. The criteria object is conformance vector TV-001 of the published PRML v0.1 suite. Its canonical form is 294 octets and its SHA-256 is 1a3466cc08ee7fb60a726ea1c4db6ecf48a9f847b9b7523bfb54b2ffaefee546, which a reader can check against the published vector without trusting this document. The full-payload Statement is 488 octets; the hash-only Statement is 197. Both are signed with EdDSA over Ed25519 under a throwaway key published alongside them. The algorithm was chosen so that the examples are byte-reproducible: EdDSA is deterministic, so a reader who regenerates them gets the same octets and can diff. The profile itself is algorithm-agnostic. Two things in this document exist because building these examples exposed them: the interim rule against asserting a payload type in hash-only mode (), and the empty unprotected header requirement above. Neither was visible from the prose alone.

Implementation Status Four byte-equivalent reference implementations of the PRML canonicalisation and digest algorithm exist, in Python, JavaScript, Go and Rust. All four were authored by the same maintainer and are validated against a shared conformance vector suite; they are not independent implementations in the sense usually meant by an interoperability report. A public PRML registry implements a hash-only sealed-commitment mechanism with application semantics similar to , using time-stamping and a public transparency log. It is not an implementation of this profile: it accepts unsigned submissions, so there is no Issuer-signed Statement, and it emits neither COSE Signed Statements nor COSE Receipts. It is therefore not a conforming Transparency Service under . No conforming implementation of this profile is known to the author at the time of writing. This status note is included so that readers do not infer a conforming deployment from the existence of this document.