Metering and billing

Every completed request is recorded in one DynamoDB transaction with two writes:

Request ledger — pk = REQUEST#<request-id>, expiring after USAGE_RETENTION_DAYS (default 90). The put is conditioned on the pk not existing, so a replayed request id cancels the transaction and the caller gets 409.
Monthly rollup — pk = CUSTOMER#<customer-id>, sk = YYYY-MM, incremented atomically. Rollups never expire.

If the metering transaction fails for any other reason, the gateway returns 503 and withholds the upstream result: a result is never released unmetered.

What is measured

Field	Meaning
`data_in_bytes`	exact HTTP request body length
`data_out_bytes`	exact response body length (stream bytes for SSE)
`model_tokens`	`b × s` for `/v1/forward`; provider `total_tokens` for LLM paths
`input_tokens` / `output_tokens`	LLM paths only, from the provider’s usage block
`request_count`, `status`, `path`, `latency_ms`	per request

The scoring backend is a scorer, not a text generator, so it has no honest “completion token” unit. model_tokens measures transformer work — the batch-size × sequence-length product the worker actually ran — and the two byte counters support ingress/egress pricing.

Streaming

For streamed chat completions the gateway forces stream_options.include_usage on the upstream request, watches the SSE stream for the final usage event, and meters after the stream ends. If the client disconnects mid-stream, the gateway keeps consuming the upstream so the usage event is still observed and recorded.

Keys

API keys are opaque bearer tokens (rtg_ prefix by convention). The keys table stores only SHA-256 digests with a customer_id and an enabled flag; plaintext is printed once at rtg-manage create-key time and never stored.