]> Independent

hello@arsentev.ai https://arsentev.ai/

agent metrics accounting large language model observability Autonomous software agents driven by large language models execute multi-step runs in which the same conversational context is retransmitted to a model on every step. The resulting resource consumption is dominated by repeated input rather than by generated output, and it is reported today in mutually incompatible, vendor-specific shapes. This document defines Agent Run Metrics, a JSON interchange format that describes the resource consumption of a single agent run and of its constituent steps, together with a small set of derived quantities whose computation is specified exactly. It states normatively that one reported step corresponds to one completed model invocation, so that the several records a runtime may write while a single response is produced are not mistaken for several invocations, and it defines how the consumption of delegated sub-runs is attributed without being counted twice. The format carries counters, timing and cost attribution only; it deliberately excludes prompt and completion content. This document also registers the associated media type and creates IANA registries for extensible enumerations.

Introduction A growing class of software systems operates as an "agent": a program that pursues a goal over a sequence of steps, at each step submitting an accumulated context to a large language model, receiving a response, and optionally invoking external tools whose results are appended to that context. The unit of work in such a system is not a single request but a run: a bounded sequence of steps sharing one accumulating context. Runs have a cost structure that differs qualitatively from that of the request/response interactions the surrounding infrastructure was designed to meter. Because the context accumulates, the input submitted at step n generally contains the input submitted at step n-1. Over a run of many steps, total input volume therefore grows super-linearly in the number of steps even when each individual step adds little. Content admitted into the context early is re-submitted on every subsequent step, so the position at which content enters a run affects total consumption as much as its size does. Practitioners consequently need per-run and per-step accounting, not merely per-request accounting, and they need it in a form that can be compared across implementations. Such accounting exists today, but only in vendor-specific shapes. Different implementations disagree on questions that are not matters of taste: whether tokens served from a provider-side cache are included in the reported input count or reported beside it; whether internal reasoning tokens that are never returned to the caller are part of the output count; whether the several records a runtime writes while one response is produced describe one invocation or several; whether the tokens of a delegated sub-run belong to that sub-run, to the run that started it, or to both; whether a retried step is one step or two; whether the totals for a run are the sum of its steps. Two reports that use the same field names can therefore describe incompatible quantities, and a consumer cannot detect the difference from the data alone. These are not questions of presentation. An accounting rule chosen wrongly changes a reported figure by a factor rather than by a percentage: a run whose journal records the progress of a response, rather than only its completion, will be reported with several times its true input if those records are summed, and a delegation tree whose parent and children both claim the children's tokens will be reported with roughly twice its true consumption. Errors of this class are silent, because the resulting report is internally consistent and no field states which rule was applied. records where such errors were observed. This document defines a JSON format, Agent Run Metrics, in which those questions are answered normatively. The format is small, content-free, and intended to be produced by an agent runtime and consumed by cost-management, capacity-planning, research and audit tooling.

Scope and Non-Goals This document specifies:

• a data model for a run, its steps, and their resource usage ();
• a JSON serialization of that data model ();
• exact definitions for a small set of derived quantities ();
• an incremental reporting profile for runs observed while in progress ();
• an OPTIONAL binding to HTTP for delivering reports to a collector ();
• the media type and IANA registries required by the above ().

This document does not specify:

• any protocol between an agent and a language model, or between an agent and its tools;
• any pricing, billing or settlement mechanism; cost fields in this format are attributions supplied by the reporter, not invoices;
• the semantic content of a run: prompts, completions, tool arguments and tool results are out of scope and, by , MUST NOT appear in a report;
• a distributed tracing system. Where a tracing system is in use, defines how to correlate with it.

Relationship to Existing Work Distributed tracing conventions such as and the semantic conventions maintained by the OpenTelemetry project describe how to represent spans of execution, including spans that invoke a language model. Those conventions address the shape of a trace and the propagation of trace identity; they do not define an interchange document for a completed run, and they do not normatively resolve the counting questions listed in . The format defined here is complementary: it is a self-contained report about one run, it can be produced without a tracing pipeline, and specifies how a report references trace identifiers when a tracing pipeline is present. This format is not a metrics protocol in the sense of a time-series exposition format. A report describes one identified run; aggregation across runs is a function of the consumer.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Run:

A bounded sequence of Steps executed by one Agent in pursuit of one goal, sharing a single accumulating Context. A Run is the unit described by a Report.

Step:

One indivisible unit of work within a Run: a single Model Invocation, a single Tool Call, or another unit whose kind is registered per .

Model Invocation:

A single request to a language model and the corresponding response, taken as one unit from the submission of the request to the completion of the response. A response delivered incrementally, or composed of several content blocks, is one Model Invocation and not several; see .

Context:

The sequence of tokens submitted as input to a Model Invocation.

Token:

The unit in which a model counts input and output. Token counts are model-specific; a Report does not assert that counts from different models are comparable.

Delegated Run:

A Run started by an Agent in the course of another Run, in pursuit of a subordinate goal and with its own Context. Such Runs are also called sub-agent Runs.

Delegation Tree:

A Run together with every Run reachable from it by delegation.

Cached Prefix:

A portion of a Context that a model provider serves from provider-side state established by an earlier Model Invocation, rather than processing afresh.

Cache Lifetime:

The period for which a provider undertakes to retain the state underlying a Cached Prefix.

Reporter:

The party that produces a Report. Typically the agent runtime.

Collector:

The party that receives and processes a Report.

Report:

One JSON document conforming to this specification.

Data Model A Report is a Run object. A Run object contains a Totals object, zero or more Step objects, and OPTIONAL descriptive and cost members. A Step object contains a Usage object when the Step consumed model resources.

Structure of a Report Usage +- cost -> Cost +- derived -> Derived +- steps [ Step ] +- index, kind, start, end +- usage -> Usage (model_invocation) +- tool -> Tool (tool_call) +- error -> Error ]]>

The Run Object A Run object has the members defined in . Members of the Run object

Member	Type	Presence	Description
version	string	REQUIRED	Format version; "1" for this specification.
run_id	string	REQUIRED	Identifier of the Run, unique within the Reporter's scope.
parent_run_id	string	OPTIONAL	run_id of the Run that delegated this Run; see .
root_run_id	string	OPTIONAL	run_id of the Run at the root of the Delegation Tree; see .
start	timestamp	REQUIRED	Instant at which the Run began.
end	timestamp	see text	Instant at which the Run reached a terminal status.
status	string	REQUIRED	One of "running", "completed", "failed", "aborted".
termination_reason	string	OPTIONAL	A registered reason; see .
agent	object	OPTIONAL	Descriptor of the producing agent; see .
models	array	OPTIONAL	Descriptors of the models used; see .
step_count	integer	REQUIRED	Number of Steps in the Run, including Steps omitted from "steps".
steps	array	OPTIONAL	Step objects; see .
totals	object	REQUIRED	Usage object aggregating this Run alone.
subtree_totals	object	OPTIONAL	Usage object aggregating this Run and its Delegated Runs; see .
cost	object	OPTIONAL	Cost object for the whole Run; see .
derived	object	OPTIONAL	Derived quantities; see .
trace_id	string	OPTIONAL	Correlation identifier; see .
labels	object	OPTIONAL	Reporter-defined string-to-string labels; see .

The "end" member MUST be present when "status" is "completed", "failed" or "aborted", and MUST NOT be present when "status" is "running". When present, "end" MUST NOT be earlier than "start". The "status" member is a closed enumeration; a Reporter MUST NOT emit a value other than the four listed, and a Collector that encounters an unrecognised value MUST treat the Report as malformed. Distinctions finer than these four values are expressed through "termination_reason", which is extensible. The "step_count" member MUST equal the total number of Steps executed in the Run. When the "steps" array is present, "step_count" MUST be greater than or equal to the length of that array; a Reporter MAY omit individual Steps from the array, for example to bound Report size, while still accounting for them in "step_count" and in "totals".

The Step Object Members of the Step object

Member	Type	Presence	Description
index	integer	REQUIRED	Zero-based position of the Step within the Run.
kind	string	REQUIRED	A registered Step kind; see .
start	timestamp	REQUIRED	Instant at which the Step began.
end	timestamp	OPTIONAL	Instant at which the Step ended.
model	string	see text	Identifier of the model invoked.
invocation_id	string	see text	Identifier of the Model Invocation; see .
usage	object	see text	Usage object for this Step.
cost	object	OPTIONAL	Cost object for this Step.
tool	object	see text	Tool object; see .
error	object	OPTIONAL	Error object; see .
child_run_id	string	see text	run_id of the Run executed by a Step of kind "delegation"; see .
retry_of	integer	OPTIONAL	index of the Step this Step retries.
span_id	string	OPTIONAL	Correlation identifier; see .

Within one Run, "index" values MUST be unique and MUST be assigned in the order in which Steps began, with no requirement that the values present in a Report be contiguous. Steps that overlap in time are permitted; "index" then reflects start order only. When "kind" is "model_invocation", the "usage" and "model" members MUST be present. When "kind" is "tool_call", the "tool" member MUST be present and the "usage" member MUST NOT be present; resources consumed by placing the tool result into the Context are attributed to the subsequent Model Invocation, where they are actually submitted. A retried attempt MUST be reported as a distinct Step with its own "index" and its own "usage", and SHOULD carry "retry_of" identifying the first attempt. Reporting a set of attempts as one Step with summed counters loses the distinction between work that advanced the Run and work that did not, and MUST NOT be done.

One Step per Completed Model Invocation A Step of kind "model_invocation" MUST describe exactly one completed Model Invocation, and a Report MUST NOT contain more than one Step describing the same Model Invocation. This requirement is stated separately, and at length, because the journals from which Reporters are commonly built do not satisfy it. A model response is typically composed of several content blocks: internal reasoning, text addressed to the caller, and requests to invoke tools. A runtime frequently appends one journal record per block, or one record per streaming update, and each such record carries a usage structure of its own. Those records are not independent measurements of independent work:

• the input counters ("input_tokens", "cache_read_tokens", "cache_write_tokens") are identical in every record of the group, because the input was submitted once;
• the output counter is a cumulative snapshot of one response as it is produced, so the records of a group form a non-decreasing sequence that ends at the final count.

Two naive treatments of such a journal therefore fail in opposite directions. Summing the records multiplies the input of the invocation by the number of records in its group and counts most of its output more than once. Retaining only the first record of each group understates the output by the ratio of the earliest snapshot to the final count, which may be an order of magnitude. Both errors are silent: the Report that results satisfies every other invariant in this document, and a Collector cannot detect either from the data. In the corpus described in , the first of these errors overstated total consumption by a factor of about 1.90 and the second halved the reported output. Accordingly:

1. A partial or intermediate usage structure, observed while a response is still being produced, is not a metric. A Reporter MUST NOT emit a Step for it and MUST NOT include its counters in "totals".
2. The "output_tokens" of a Step of kind "model_invocation" MUST be the count for the whole completed response. It MUST NOT be a snapshot taken before the response ended, and MUST NOT be the sum of the snapshots observed while it was produced.
3. A response composed of several content blocks MUST be reported as one Step whose counters cover all of its blocks. Reasoning blocks are accounted for within that Step by "reasoning_tokens", and requests to invoke tools are part of its "output_tokens"; the Tool Calls to which those requests give rise are separate Steps of kind "tool_call".
4. A Reporter deriving Reports from a source that emits several records per Model Invocation MUST group those records by an identifier of the invocation and emit one Step per group. It MUST NOT group by timestamp, by model identifier or by adjacency in the source, because concurrent invocations interleave and such grouping silently merges or splits them.

A Step of kind "model_invocation" SHOULD carry "invocation_id", a string identifying the Model Invocation and unique within the Run. Where the source of a Report distinguishes invocations by such an identifier, the Reporter MUST use it both as the grouping key required above and as the value of this member, so that a Collector can verify the grouping instead of trusting it. A Reporter MUST NOT emit two Steps of one Run with the same "invocation_id". A Collector that receives a Report containing two such Steps MUST treat the Report as malformed rather than silently discard one of them, since it cannot know which carries the final counters. An invocation that never completed, because it failed, was cancelled or was abandoned, MAY be reported as a Step carrying an Error object together with whatever counters the provider metered for it. Such a Step is a completed unit of accounting even though it is not a completed response. The prohibition above concerns snapshots of an invocation that is still producing output, not invocations that ended early.

The Usage Object The Usage object is the core of this format, and the definitions in this section are the ones on which interoperability depends. Members of the Usage object

Member	Type	Presence	Description
input_tokens	integer	REQUIRED	Total tokens submitted as input.
output_tokens	integer	REQUIRED	Total tokens produced by the model.
cache_read_tokens	integer	OPTIONAL	Input tokens served from a Cached Prefix.
cache_write_tokens	integer	OPTIONAL	Input tokens stored into provider-side cache state.
cache_writes	array	OPTIONAL	Breakdown of "cache_write_tokens" by Cache Lifetime; see .
reasoning_tokens	integer	OPTIONAL	Output tokens not returned to the caller.
billable_input_tokens	integer	OPTIONAL	Input tokens the provider metered.
billable_output_tokens	integer	OPTIONAL	Output tokens the provider metered.

All members of a Usage object MUST be non-negative integers. The following invariants are normative. A Reporter MUST NOT emit a Usage object that violates them, and a Collector MUST treat a Report containing such an object as malformed:

1. "input_tokens" is the count of all tokens submitted as input, whether or not they were served from a Cached Prefix. "cache_read_tokens" is a subset of "input_tokens" and therefore MUST be less than or equal to it. A Reporter MUST NOT report cached tokens beside "input_tokens" as a disjoint quantity.
2. "output_tokens" is the count of all tokens produced, including tokens consumed internally by the model and never returned to the caller. "reasoning_tokens" is a subset of "output_tokens" and therefore MUST be less than or equal to it.
3. "cache_write_tokens" counts tokens that were submitted as input and, in the course of the same Model Invocation, stored into provider-side cache state for reuse by later invocations; it is likewise a subset of "input_tokens". A token that was served from a Cached Prefix was not processed and stored afresh in the same invocation, so "cache_read_tokens" and "cache_write_tokens" denote disjoint subsets of "input_tokens" and their sum MUST be less than or equal to "input_tokens". A Reporter MUST NOT report tokens stored into cache state beside "input_tokens" as a disjoint quantity, and MUST NOT report them a second time under "cache_read_tokens" merely because a later invocation read them.
4. The "billable_" members carry the counts a provider used for metering, which may differ from the physical counts because of discounts applied to cached tokens, rounding, or minimum charges. They are unconstrained relative to the physical counts. A Reporter that has no metering information MUST omit them rather than copy the physical counts, so that a Collector can distinguish "metered and equal" from "not known".

The distinction between the three ways in which an input token can be handled, namely processed afresh, served from a Cached Prefix, or processed and stored into cache state, is not a refinement of interest only to implementers. Providers price the three differently, commonly charging a premium for tokens stored and a discount for tokens served from store, so two Runs with identical "input_tokens" can differ in cost by a large factor. In the corpus described in , tokens served from a Cached Prefix accounted for about 56 percent of attributed cost and tokens stored into cache state for about 27 percent, leaving less than a fifth for input processed afresh and for all output together; storing into cache was thus the second largest component of cost rather than a marginal one. A Reporter that omits "cache_read_tokens" and "cache_write_tokens" leaves the greater part of the cost of a Run unattributable, and SHOULD emit both whenever the provider reports them. When a Run's "steps" array contains every Step of the Run, the "totals" object MUST be, member by member, the sum of the corresponding members of every Step's Usage object, with the exception of "cache_writes", whose aggregation is defined in , and a member MUST be omitted from "totals" if it is absent from every Step. When the "steps" array is incomplete, "totals" MUST still account for the whole Run, including omitted Steps.

Cache Lifetime Provider-side cache state is retained for a bounded period, and a provider may offer several such periods at different prices. Two Model Invocations that stored the same number of tokens may therefore have been charged differently, and a single "cache_write_tokens" figure cannot distinguish them. A Usage object MAY for that reason carry "cache_writes", an array of objects each having the REQUIRED members "lifetime" and "tokens". The "lifetime" member MUST be a string conforming to the "duration" rule of Appendix A of , giving the nominal retention period that the Reporter requested or that the provider applied, for example "PT5M" or "PT1H". The value states a nominal period and not an observed one; whether the state in fact survived that long is not reported. The "tokens" member MUST be a non-negative integer. One "lifetime" value MUST NOT appear in more than one element of the same array. The sum of the "tokens" members of "cache_writes" MUST equal the "cache_write_tokens" member of the same Usage object. This does not always hold in the sources from which Reporters are built: in the corpus described in , a small number of records carried per-lifetime counters whose sum differed from the aggregate counter in the same record. A Reporter whose source does not reconcile MUST omit "cache_writes" for the affected Step. It MUST NOT adjust either figure so that they agree, and MUST NOT substitute the sum of the per-lifetime counters for "cache_write_tokens", because a discrepancy is evidence that at least one of the two figures is wrong, and either repair would assert an agreement that was never observed. A Collector MUST treat as malformed a Report in which "cache_writes" is present and does not sum to "cache_write_tokens". Where "cache_writes" is absent, a Collector MUST NOT assume any particular lifetime. Where the Usage objects of Steps carry "cache_writes", a "totals" or "subtree_totals" object that carries it MUST contain one element for each distinct "lifetime" that appears, whose "tokens" is the sum of the "tokens" reported for that lifetime; this is the aggregation rule referred to in . There is deliberately no corresponding breakdown of "cache_read_tokens". A read is served from whatever state exists at the moment of the read, and attributing it to the lifetime with which that state was created would require knowledge the reader does not have.

Delegated Runs and Aggregation An Agent commonly pursues part of its goal by starting a subordinate Run with its own Context, and that Run may itself delegate further. A Delegated Run is a Run in the sense of this document: it has its own "run_id", its own Steps and its own "totals", and it is described by its own Report. In the corpus described in , slightly more than half of all Model Invocations occurred in Delegated Runs rather than in the Runs that started them, so a treatment that ignored delegation would describe a minority of the work. The relationship is expressed in three places, and a Reporter that observes both sides SHOULD populate all three:

• the child Report's "parent_run_id", naming the Run that delegated it;
• the child Report's "root_run_id", naming the Run at the root of the Delegation Tree. A Run that emits "root_run_id" and has no parent MUST set it equal to its own "run_id"; a Run that emits it and has a parent MUST set it to the "root_run_id" of that parent. The member exists so that a Collector can select a whole Tree without walking it edge by edge;
• in the parent, a Step of kind "delegation" spanning the interval during which the child ran, carrying "child_run_id" where the parent knows the child's identifier.

The counting rule is that "totals" is per-Run and is not transitive. The "totals" of a Run MUST account for the Steps of that Run alone and MUST NOT include any resource consumed by a Delegated Run, whether or not that Run is separately reported. A Step of kind "delegation" accordingly carries no Usage object, as requires: the delegating Run submits no Context of its own while the child runs. The resources consumed in placing the child's result into the parent's Context are attributed, like any other addition to a Context, to the parent's next Model Invocation. A Run MAY additionally carry "subtree_totals", a Usage object accounting for that Run together with every Run reachable from it by delegation. When present, each member of "subtree_totals" MUST be greater than or equal to the corresponding member of "totals". A Reporter that cannot observe the whole Tree MUST omit "subtree_totals" rather than emit a partial figure under that name. A Collector computing the consumption of a Delegation Tree MUST use exactly one of two methods: sum the "totals" of every Run of the Tree, or take the "subtree_totals" of its root. It MUST NOT combine them, and in particular MUST NOT add a Run's "subtree_totals" to the "totals" of that Run's descendants; either mistake counts the delegated work twice. Where Reports arrive independently, a Collector SHOULD prefer summing "totals", which remains well defined when the Tree is incomplete, and SHOULD treat "subtree_totals" as an assertion by the Reporter to be checked once the Tree is complete. Derived quantities are per-Run in the same sense. In particular "input_amplification" () is computed over the Steps of one Run; the value reported for a parent says nothing about its children, and the values of a Tree cannot be combined into a value for the Tree.

The Cost Object Members of the Cost object

Member	Type	Presence	Description
currency	string	REQUIRED	Three-letter alphabetic currency code .
amount	string	REQUIRED	Decimal amount; see below.
basis	string	REQUIRED	One of "metered", "list_price", "estimated".
pricing_ref	string	OPTIONAL	URI identifying the price schedule used.

The "amount" member MUST be a JSON string matching the ABNF rule: A string is used rather than a JSON number so that the value survives parsers that map numbers to binary floating point. A Collector MUST NOT convert an "amount" to a binary floating point value before comparison or summation. The "basis" member states the provenance of the figure. "metered" means the amount was obtained from the provider's metering of this Run or Step. "list_price" means it was computed by the Reporter from a published price schedule. "estimated" means it was computed by any other means. A Reporter MUST NOT report "metered" for a figure it computed itself. Cost figures are attributions made by the Reporter. They are not invoices and MUST NOT be treated as authoritative for settlement.

The Tool Object A Tool object describes a Step of kind "tool_call". It has the REQUIRED member "name", a string naming the tool within the Reporter's scope, and the REQUIRED member "outcome", whose value is one of "ok", "error" or "timeout". It MAY carry "bytes_in" and "bytes_out", non-negative integers giving the octet length of the arguments passed to and the result returned from the tool. The "name" member identifies a class of tool, not an instance, and MUST NOT encode arguments, targets, identifiers or any other content of the call; see .

The Error Object An Error object has the REQUIRED member "type", a short lowercase token classifying the failure, and the OPTIONAL boolean member "retryable". The "type" member MUST NOT carry a provider's free-text error message, which may quote the Context; see .

The Agent Object An Agent object has the OPTIONAL string members "name" and "version", identifying the software that executed the Run. These members describe software, not the party operating it; see .

The Model Object A Model object has the REQUIRED string member "id" and the OPTIONAL string member "provider". It MAY carry "context_window", a positive integer giving the maximum number of input tokens the model accepts. Where reported, "context_window" allows a Collector to relate a Step's "input_tokens" to the limit that Step was approaching, which is the quantity of operational interest.

The Labels Object The "labels" member carries Reporter-defined dimensions along which a Collector may group Runs, such as a deployment environment or a workload class. Its value MUST be a JSON object whose members all have string values. Label keys and values MUST NOT carry content from the Run and SHOULD NOT carry identifiers of natural persons; see .

Derived Quantities The quantities in this section are computable from the members already defined. They are specified here because they are the quantities in which the behaviour of a Run is usually discussed, because they are easy to compute in mutually incompatible ways, and because a Collector that receives an incomplete "steps" array cannot recompute them itself. A Reporter MAY omit the "derived" object entirely; if it includes any member of that object, the member MUST be computed exactly as specified below.

Input Amplification Let S be the set of Steps of kind "model_invocation" in the Run, and let i(s) be the "input_tokens" of Step s. The member "input_amplification" is defined as: It is a JSON number, and it MUST be omitted when S is empty or when the denominator is zero. Its value is at least 1. It expresses how many times the largest Context reached in the Run was, in effect, paid for. A Run of many Steps whose Context grows steadily has an input amplification of roughly half its Step count; a Run whose Context is fully rebuilt at each Step has one close to its Step count; a Run that does its work in a single Step has exactly 1. This quantity deliberately uses the largest Context reached, not the final Context, so that Runs that compact or truncate their Context remain comparable with Runs that do not. Observed values span orders of magnitude, which is the reason the quantity is worth reporting at all: in the corpus described in , the median Run had an input amplification of about 24, the ninetieth percentile about 125, and the largest value observed exceeded 4000. A single aggregate token count does not separate such Runs from one another. Because the quantity is computed over the Steps of one Run, a delegating Run and each of its Delegated Runs yield separate observations; see .

Cache Hit Ratio The member "cache_hit_ratio" is defined as the "cache_read_tokens" of the Run's "totals" divided by the "input_tokens" of the Run's "totals". It is a JSON number in the interval from 0 to 1 inclusive, and it MUST be omitted when "cache_read_tokens" is absent from "totals" or when "input_tokens" is zero.

Cache Write Ratio The member "cache_write_ratio" is defined as the "cache_write_tokens" of the Run's "totals" divided by the "input_tokens" of the Run's "totals". It is a JSON number in the interval from 0 to 1 inclusive, and it MUST be omitted when "cache_write_tokens" is absent from "totals" or when "input_tokens" is zero. It is specified alongside "cache_hit_ratio" because the two answer different questions and neither can be inferred from the other. A high "cache_hit_ratio" says that the Run re-submitted a Context that was already stored; a high "cache_write_ratio" says that the Run kept establishing new cache state, which a provider commonly charges at a premium. A Run can exhibit both. By the disjointness required in , the two ratios cannot sum to more than 1.

Output Share The member "output_share" is defined as the "output_tokens" of the Run's "totals" divided by the sum of "input_tokens" and "output_tokens" of the Run's "totals". It is a JSON number in the interval from 0 to 1 inclusive, and it MUST be omitted when that sum is zero.

Step Efficiency The member "advancing_step_ratio" is defined as the number of Steps that do not carry a "retry_of" member and whose "error" member is absent, divided by "step_count". It is a JSON number in the interval from 0 to 1 inclusive. A Reporter MUST NOT emit this member when its "steps" array is incomplete, because the numerator is then not determinable from the Report.

Numeric Precision The derived members are JSON numbers and are inherently approximate. A Reporter SHOULD NOT emit more than six significant digits. A Collector MUST NOT rely on a derived member being reproducible bit-for-bit from the Report's own counters, and SHOULD recompute derived quantities from counters when the "steps" array is complete.

Serialization A Report MUST be a JSON object conforming to the restricted profile of . In particular, a Report MUST be encoded in UTF-8, its objects MUST NOT contain duplicate member names, and integers MUST be within the range that can be represented exactly, that is from -(2^53)+1 to (2^53)-1 inclusive. Timestamps MUST be strings conforming to the "date-time" production of . They MUST use the "Z" time offset and SHOULD include at least milliseconds of fractional seconds. Local offsets are excluded because they disclose the operator's location without carrying information the format needs. Identifier members, namely "run_id", "parent_run_id", "root_run_id", "child_run_id", "invocation_id" and the "name" members of the Agent and Tool objects, MUST be non-empty strings of at most 128 characters. A Collector MAY reject a Report whose identifiers exceed that length. A machine-readable schema for the format is expected to accompany a future revision of this document. Where such a schema and the text of this document disagree, the text governs; the subset invariants of and the computation rules of are not expressible in schema languages in common use.

Extensibility A Collector MUST ignore any object member it does not recognise, and MUST NOT treat its presence as an error, except where this document specifies otherwise for closed enumerations. A Reporter that adds members not defined in this document or in a registry established by it MUST name them with a prefix of the form "x-" followed by a label that is unlikely to collide, such as a reversed domain name. A Reporter MUST NOT use an unprefixed member name for a purpose other than the one specified here. The "version" member identifies the format, not the Reporter. A Reporter MUST emit "1" while conforming to this specification. A Collector that receives a "version" it does not recognise SHOULD reject the Report rather than interpret members whose meaning may have changed.

Correlation with Tracing Systems When the Reporter participates in a tracing system that uses the identifiers of , it SHOULD populate the Run's "trace_id" member with the lowercase hexadecimal encoding of the 16-octet trace identifier, and each Step's "span_id" member with the lowercase hexadecimal encoding of the 8-octet span identifier of the span covering that Step. These members exist to permit a Collector to join a Report to a trace; this document does not otherwise depend on any tracing system, and a Reporter that does not participate in one MUST omit them rather than invent values.

Incremental Reporting A Run may be long-lived, and a Collector may need to observe it before it terminates. A Reporter MAY therefore emit more than one Report for the same "run_id". Every Report for a Run MUST be self-contained: its "totals" MUST account for the Run from its "start" up to the moment of the Report, not for the interval since the previous Report. A Collector therefore replaces rather than accumulates. To let a Collector order Reports that arrive out of order, a Reporter emitting more than one Report for a Run MUST include the member "revision", a non-negative integer that starts at 0 and increases by 1 for each subsequent Report about the same "run_id". A Collector MUST discard a Report whose "revision" is not greater than that of a Report it has already accepted for the same "run_id", and MUST treat this discard as normal operation rather than as an error. At most one Report for a given "run_id" may have a terminal "status". A Collector that receives a second Report with a terminal status for a "run_id" for which it has already accepted one MUST discard the later Report, whatever its "revision". A sequence of Reports MAY be serialized as JSON text sequences , one Report per element. Whole-Run Reports for distinct Runs MAY be batched the same way. A batch MUST NOT be represented as a JSON array of Reports, because a consumer cannot then process the batch incrementally.

HTTP Binding This section defines an OPTIONAL binding for delivering Reports to a Collector over HTTP . Implementations MAY convey Reports by other means; the format does not depend on this binding. A Reporter delivers a Report by issuing a POST request to a Collector URI configured out of band, with a "Content-Type" header field of "application/agent-run-metrics+json". A batch is delivered with a "Content-Type" of "application/agent-run-metrics-seq+json". The request MUST be sent over a connection providing confidentiality and integrity; in practice this means TLS or a protocol offering equivalent protection. A Collector MUST NOT accept Reports over an unprotected connection. A Collector that has accepted a Report for processing SHOULD respond with 202 (Accepted) and an empty body. A Collector that rejects a Report SHOULD respond with a 4xx status code and a problem details document describing the reason. Delivery is idempotent by construction: the pair of "run_id" and "revision" identifies a Report, and a Collector that receives the same pair twice MUST treat the second delivery as a duplicate and respond as it did to the first. A Reporter MAY therefore retry a request whose outcome it does not know, and SHOULD space retries using exponential backoff with jitter. A Collector MUST impose a limit on the size of a Report it will accept and SHOULD respond with 413 (Content Too Large) when that limit is exceeded. A Reporter that receives 413 SHOULD retry with an abbreviated "steps" array as permitted by , rather than dropping the Report.

Privacy Considerations

Exclusion of Content A Report MUST NOT contain any part of the Context, of a model response, of a tool's arguments, or of a tool's result. This prohibition is absolute and applies to every member of every object defined here, including free-form members such as "labels", the Agent object's "name", the Tool object's "name" and the Error object's "type". A Reporter that wishes to convey content operates outside this format. The prohibition exists because a metrics pipeline typically has a wider audience, a longer retention period and weaker access control than the system that produced the Run. Content that leaks into a Report inherits the pipeline's exposure, not the Run's.

Inference from Counters Counters are not content, but they are not free of information about content either. The length of a document supplied to a Run is closely related to the increase in "input_tokens" at the Step that first submitted it; the presence of a Step of kind "tool_call" with a particular "name" discloses that the Run used that class of tool; timing members disclose when a Run was executed and, in aggregate, an operator's working hours. Where a Run's inputs are drawn from a small or guessable set, an observer holding candidate inputs can compare their token lengths against a Report and identify which was used. A Reporter SHOULD therefore treat Reports as confidential to the same degree as the workload they describe, and SHOULD NOT publish per-Step Reports for Runs whose inputs are sensitive. Publishing only Run-level "totals" and "derived" members substantially reduces, but does not eliminate, this exposure.

Identifiers The "run_id" member is opaque to this format and MUST NOT be derived from an identifier of a natural person, a session, an account or a request that the Collector could correlate with one. A Reporter SHOULD generate "run_id" values that carry no structure, for example as UUIDs . A Report describes a Run, and a Run is usually initiated on behalf of a person. Retaining Reports therefore constitutes processing of data about that person's activity even though no personal data appears in the Report itself. Operators SHOULD apply retention limits to Reports and SHOULD aggregate them once the individual Runs are no longer operationally relevant.

Security Considerations The privacy considerations in are also security considerations and are not repeated here.

Trust in the Reporter Every value in a Report is asserted by the Reporter. This format provides no means for a Collector to verify a count, a timestamp or a cost against the provider that actually served the Run. A Collector MUST NOT treat a Report as evidence of what a Run consumed; it is a statement by the Reporter about what the Run consumed. Where an independent guarantee is required, it must come from a signature over the Report or from reconciliation against the provider's own metering, neither of which is specified here. Where Reports drive chargeback, quota enforcement or capacity decisions, a Reporter has an incentive to under-report and, in adversarial multi-tenant settings, a neighbour has an incentive to over-report on another tenant's behalf. A Collector MUST authenticate Reporters and MUST reject a Report that claims a "run_id" outside the authenticated Reporter's namespace. Mechanisms for authentication are out of scope; the HTTP binding of can carry any of the standard HTTP authentication schemes.

Resource Exhaustion at the Collector A Report's "steps" array is unbounded in the format, and a Run may legitimately contain a very large number of Steps. A Collector that parses Reports of unbounded size, or that retains one entry per "run_id" without limit, can be exhausted by a Reporter that emits many Runs or one very large Run. A Collector MUST enforce limits on Report size, on the number of Steps it will process, and on the number of distinct "run_id" values it will track per Reporter, and SHOULD apply rate limits per authenticated Reporter. The incremental profile of permits an unbounded number of Reports per "run_id". A Collector SHOULD bound the "revision" values it will accept for a single Run and SHOULD expire tracking state for Runs that have neither terminated nor been updated within an operator-configured interval.

Parsing Reports arrive from parties that may be compromised even when they are authenticated. A Collector MUST validate a Report against the constraints of before acting on it, including the integer range restriction, which exists to prevent silent truncation in implementations that map JSON numbers to binary floating point. Because the "amount" member is a string, a Collector MUST bound its length before parsing it as a decimal, so that an arbitrarily long digit string cannot drive unbounded work in an arbitrary-precision library.

Delegation The "parent_run_id" member permits a Reporter to attribute a Run to another Run, potentially one reported by a different Reporter. A Collector MUST NOT allow a Reporter to attach a Run to a parent outside that Reporter's namespace without an authorization decision of its own, because doing so would let one party inflate another's accounted consumption. A Report may carry "subtree_totals" as well as "totals" (), and the two are aggregated by mutually exclusive methods. A Collector that combines them counts delegated consumption twice. Where accounted consumption allocates cost or enforces a budget, a Reporter that can influence which method a Collector applies can inflate or deflate the accounted figure without emitting any value that an invariant of this document forbids. A Collector SHOULD therefore fix its aggregation method by policy and SHOULD NOT infer it from the members a Report happens to contain.

IANA Considerations

Media Type application/agent-run-metrics+json IANA is requested to register the following media type in the "Media Types" registry, per .

Type name:

application

Subtype name:

agent-run-metrics+json

Required parameters:

N/A

Optional parameters:

N/A

Encoding considerations:

binary; the content is JSON text encoded in UTF-8

Security considerations:

See of RFC XXXX

Interoperability considerations:

See of RFC XXXX

Published specification:

RFC XXXX

Applications that use this media type:

Agent runtimes, cost-management, capacity-planning and audit tooling

Fragment identifier considerations:

As specified for "application/json" in

Additional information:

Deprecated alias names for this type: N/A Magic number(s): N/A File extension(s): .json Macintosh file type code(s): TEXT

Person and email address to contact for further information:

IETF (iesg@ietf.org)

Intended usage:

COMMON

Restrictions on usage:

None

Author:

See the "Authors' Addresses" section of RFC XXXX

Change controller:

IETF

Media Type application/agent-run-metrics-seq+json IANA is requested to register a second media type, identical to that of except as follows.

Subtype name:

agent-run-metrics-seq+json

Encoding considerations:

binary; the content is a JSON text sequence as specified in , each element of which is a Report

Interoperability considerations:

See of RFC XXXX

File extension(s):

.jsonseq

Agent Run Metrics Termination Reasons Registry IANA is requested to create a new registry group named "Agent Run Metrics", and within it a registry named "Agent Run Metrics Termination Reasons". The registration policy is Specification Required . The designated expert is directed to confirm that a proposed reason denotes a cause of termination that is distinguishable from every registered reason, that its name uses only lowercase ASCII letters, digits and the underscore character, and that the specification defines the reason without reference to any single implementation. Each entry has the fields: Reason (the string value), Description, Terminal Statuses (the subset of "completed", "failed" and "aborted" with which the reason may appear), Reference, and Change Controller. The registry is to be populated with the initial entries in . The Reference for each is RFC XXXX and the Change Controller for each is IETF. Initial Termination Reasons

Reason	Description	Terminal Statuses
goal_reached	The Agent determined the goal was satisfied.	completed
step_limit	A configured limit on the number of Steps was reached.	completed, aborted
budget_exhausted	A configured limit on tokens or cost was reached.	completed, aborted
context_limit	The Context reached the model's context window.	failed, aborted
time_limit	A configured wall-clock limit was reached.	completed, aborted
tool_failure	A tool failed in a way the Agent could not recover from.	failed
model_error	A model invocation failed and could not be retried successfully.	failed
policy_refusal	The Run stopped because a policy declined to proceed.	completed, aborted
operator_abort	An operator stopped the Run.	aborted
internal_error	The Agent failed for a reason not covered above.	failed

Agent Run Metrics Step Kinds Registry IANA is requested to create, within the "Agent Run Metrics" registry group, a registry named "Agent Run Metrics Step Kinds". The registration policy is Specification Required . The designated expert is directed to confirm that a proposed kind describes a unit of work that consumes resources distinguishable from those of every registered kind, and to reject kinds that merely subdivide an existing kind along a dimension better expressed through "labels". Each entry has the fields: Kind (the string value), Description, Required Members (members of the Step object that MUST be present for this kind), Prohibited Members, Reference, and Change Controller. The registry is to be populated with the initial entries in . The Reference for each is RFC XXXX and the Change Controller for each is IETF. Initial Step Kinds

Kind	Description	Required	Prohibited
model_invocation	A single request to a language model and its response.	usage, model	tool
tool_call	A single invocation of a tool external to the model.	tool	usage
retrieval	A search over a corpus whose results are placed into the Context.	tool	usage
delegation	Execution of a child Run, reported separately.	(none)	usage
human_input	A pause awaiting input from a person.	(none)	usage, tool
compaction	A transformation that reduces the Context in place.	(none)	tool

The "delegation" kind marks the interval during which a subordinate Run executed. It prohibits "usage" because the delegating Run submits no Context of its own while the child runs; the child's consumption is reported by the child, as requires. A Step of this kind SHOULD carry "child_run_id". The "compaction" kind permits "usage", because a Context is commonly compacted by invoking a model on it; the resources so consumed are attributed to the compaction rather than to the Step that follows it.

Implementation Status and Empirical Basis At the time of writing this document describes a proposed format, and the author is aware of no implementation of it. The examples in are illustrative constructions; none of the values in them was measured from any system. The requirements in , and were not arrived at from the shape of the format alone. They follow from an analysis of the execution journals of one agent runtime, covering 722 Runs, some 150,000 Model Invocations and roughly 34 billion tokens, collected by a single practitioner. That corpus, its collection method and its analysis are published separately; this document does not restate them. It is a single-practitioner corpus from one runtime and one provider, and no claim is made that its magnitudes generalise to other deployments. What it establishes is not the values but the existence of the failure modes the values illustrate:

• the runtime wrote several journal records per Model Invocation, one per content block of the response, whose input counters repeated and whose output counter was a cumulative snapshot. Summing them overstated consumption by a factor of about 1.90; taking the first record of each group understated output by about half ();
• slightly more than half of all Model Invocations occurred in Delegated Runs rather than in the Runs that started them, so the rule against attributing a child's tokens to its parent as well as to itself governs the majority of the work ();
• tokens stored into provider-side cache state accounted for about 27 percent of attributed cost and tokens served from that state for about 56 percent, so cache accounting rather than fresh input dominated the cost of a Run ();
• per-lifetime cache counters and the aggregate cache counter of the same record disagreed in a small number of records, which is why requires the breakdown to be omitted rather than reconciled ();
• input amplification had a median of about 24, a ninetieth percentile of about 125 and a maximum above 4000 ().

These figures are recorded here as observations of one corpus, to explain why the corresponding requirements exist. They are not normative, and a Collector MUST NOT use them as expected values against which to validate a Report.

References Normative References International Organization for Standardization Informative References World Wide Web Consortium W3C Recommendation OpenTelemetry Authors

Examples The Reports in this appendix are illustrative. They were constructed to demonstrate the format and the invariants of ; the values in them are not measurements of any system, and no conclusion should be drawn from their magnitudes.

Minimal Report The smallest conforming Report omits every OPTIONAL member.

Report with Steps, Cache and Derived Quantities The following illustrative Report shows a Run of four Steps in which the Context accumulates across three Model Invocations separated by one Tool Call. Note that "cache_read_tokens" and "cache_write_tokens" are in every case disjoint subsets of "input_tokens", that each Model Invocation is one Step bearing one "invocation_id" however many content blocks its response contained, and that "totals" is the member-by-member sum of the Steps that carry a Usage object, "cache_writes" being aggregated per lifetime. The "cost" object above carries a zero "amount" with a basis of "estimated" solely to exhibit the object's shape; it is not a price. The third Model Invocation illustrates the rule of . Its response consisted of a reasoning block and a text block, and the runtime that produced it wrote a record for each. The Report contains one Step whose "output_tokens" is 905, the count for the whole response, of which 520 are attributed to reasoning. A Reporter that had emitted one Step per record would have reported 30600 input tokens for this invocation instead of 15300.

A Run and the Run It Delegates The two Reports below describe one Delegation Tree. The parent's Step of index 1 spans the interval during which the child ran and carries no Usage object; the child reports its own consumption. The parent also asserts "subtree_totals", which equals the member-by-member sum of the two "totals" objects. A Collector that added "subtree_totals" to the child's "totals" would count the child twice; see . The child reports itself independently. It omits its "steps" array, which "step_count" and "totals" still account for.

Incremental Report A Reporter observing a Run in progress emits a self-contained Report with a non-terminal "status", no "end", and a "revision".

Acknowledgements This document has not yet benefited from review. The author welcomes comment.