Deployment

AWS backing services

aws cloudformation deploy --template-file infra/infra-fly.yaml \
  --stack-name relativedb-gateway-core --capabilities CAPABILITY_NAMED_IAM \
  --profile personal --region us-west-2

The stack creates the Cloud Map registry, both DynamoDB tables, and a least-privilege IAM role trusted only by the relativedb-gateway Fly app through short-lived OIDC credentials — no permanent AWS key is stored at Fly.

Note

The Fly app name, IAM role, and table names predate this repository and are kept as-is: the OIDC trust condition binds to the Fly app name, so renaming any of them breaks the deployed stack.

Fly

flyctl deploy --remote-only --ha=false
flyctl certs add api.relativedb.com

Fly terminates TLS on one 512 MB shared-cpu-1x Machine in San Jose. The checked-in fly.toml allows five-minute upstream requests and uses connection-based concurrency limits, which accommodates long inference polls. The health check probes /gateway/health every 15 seconds. rtg-serve finishes in-flight requests on SIGTERM, so Fly deploys drain cleanly.

Workers

Register each worker with a stable instance id and an externally reachable URL; deregister before stopping it:

rtg-manage register-worker --service-id srv-afwgq7ghit7s35ko \
  --instance-id vast-123 --url https://worker.example:8500
rtg-manage list-workers --namespace workers.relativedb.local \
  --service inference
rtg-manage unregister-worker --service-id srv-afwgq7ghit7s35ko \
  --instance-id vast-123

Only register HTTPS URLs unless the worker is reached through a private network — a public plain-HTTP worker would expose request data in transit. A stale registration (a worker that died without deregistering) costs one connection attempt per request until removed; the gateway logs each failed attempt with the worker URL.

scripts/vast-test-worker.sh starts a stand-in worker on a GPU box — it serves /health (reporting the GPU) and a /v1/forward stub — for wiring tests without a real scorer.

Keys

rtg-manage create-key --table relativedb-gateway-api-keys --customer acme
rtg-manage revoke-key --table relativedb-gateway-api-keys --key rtg_...

Revocation takes effect within AUTH_CACHE_SECONDS (default 30 s).