Relational Transformers Gateway

The public control and billing layer in front of Relational Transformers inference workers. One small persistent server authenticates API keys, discovers GPU workers through AWS Cloud Map, proxies scoring and OpenAI-compatible LLM traffic (including SSE streaming), and atomically records per-customer usage in DynamoDB.

client -> Fly TLS/proxy -> gateway -> Cloud Map -> workers
                                   -> OpenAI-compatible provider

Design in one paragraph: Gateway.handle(method, path, headers, body) is a transport-agnostic core with injectable AWS clients; the HTTP server is a thin wrapper that adds SSE streaming. Every completed request is metered in a single DynamoDB transaction — an expiring request-ledger row conditioned on a new request id plus a monthly customer rollup — so a replayed request id gets a 409 instead of double-billing, and a result is never released unmetered.

Usage

• Quickstart
  • Install
  • Run locally
  • Mint a key and score
  • Endpoints
• Configuration
  • Required
  • Local model execution
  • Optional
  • Semantics worth knowing
• Deployment
  • AWS backing services
  • Fly
  • Workers
  • Keys
• Metering and billing
  • What is measured
  • Streaming
  • Keys

Project

• Benchmarks
• Testing

Package Reference

• API Reference