219 lines · 15.0 KB

Usage metering and billing

All billable usage is recorded through src/server/metering/. Every event is billed to one account (a user, an org, or a scratch: account for anonymous usage) and never to two.

Design

  • Source of truth: D1. usage_events holds one row per event, keyed by its idempotency key. AFTER INSERT triggers fold each new row into usage_periods (per account, meter and month), usage_repo_periods and usage_days in the same write. A retried event hits INSERT OR IGNORE on the same key, so it inserts nothing and the triggers never fire: retries can never double count.
  • Analytics Engine (USAGE binding, dataset gitfor_usage) gets a data point for every newly inserted event (index = account id; blobs = meter, period, repo id, idempotency key, attributes; double = quantity). Use it for high-cardinality exploration and the site-admin dashboards. It is best-effort and is never used for billing.
  • Hard caps use D1, not a Durable Object. A reservation is one D1 batch, which is a single transaction. The batch inserts the reservation row (with a random nonce) only when quantity + reserved + q <= cap, then adds the hold only if that nonce landed. It does not rely on changes(). D1 serializes writes, so this conditional check is atomic. It cannot be overshot by concurrent reservations, and it needs no extra binding or per-account DO hop. A DO counter would only pay off at more than about 1k reservations per second per account, and hard-capped meters (AI calls, editor seconds, scratch sites) are nowhere near that rate.
  • Cron (runMeteringCron, every 5 minutes) releases expired reservations, snapshots storage gauges, sends budget alerts, and prunes events older than 400 days. The aggregates are kept forever.
  • Periods are UTC calendar months (YYYY-MM) and are taken from the event time. An event at 23:59:59 on the last day of a month counts toward that month, even if it is recorded later.
  • Precision. Compute meters are in seconds, and fractions are allowed. Never round up to the minute. Pass (endMs - startMs) / 1000.

Meters (src/lib/meters.ts)

Meter Unit Kind Who records it
storage.artifacts bytes gauge cron, from repo_storage_estimates (pack bytes received on push)
storage.lfs bytes gauge cron, from refcounted lfs_usage
storage.ci_cache bytes gauge cron, from action_caches
storage.sites bytes gauge cron, from active site_deployments
storage.r2 bytes gauge setGauge by the owning subsystem
compute.runner seconds counter Runner DO / ci (metering)
compute.runner_large seconds counter same
compute.runner_xlarge seconds counter same
compute.runner_self_hosted seconds counter ci on job completion; free
compute.editor seconds counter editor containers (container-meter.ts, reserve/commit); hard cap
compute.site_requests count counter sites dispatch (orangered-llama)
compute.site_cpu ms counter sites dispatch (orangered-llama)
compute.workflow_steps count counter ci; free, informational
network.lfs bytes counter LFS download (metering)
network.sites bytes counter sites dispatch (orangered-llama)
network.downloads bytes counter raw and archive (metering)
ai.calls count counter AI Gateway path (meteredAiCall); hard cap
ai.tokens tokens counter same; attribute model
ops.integration_deliveries count counter integrations (lightblue-moth)
ops.mcp_calls count counter MCP hub (slateblue-rabbit)
ops.git_ops count counter git proxy (metering); attribute op = clone, fetch or push
ops.git_bytes bytes counter git proxy (metering); attribute direction = in or out

API

import { recordUsage, recordUsageSoon, setGauge, claimScratchUsage } from "~/server/metering/record";
import {
  reserveUsage,
  commitUsage,
  releaseUsage,
  assertWithinQuota,
  QuotaExceededError,
} from "~/server/metering/quota";
import {
  repoOwnerAccount,
  ciRunAccount,
  tokenAccount,
  scratchSiteAccount,
  scratchIpAccount,
} from "~/server/metering/attribution";

recordUsage(input): Promise<boolean>

await recordUsage({
  accountId, // billed account (use the attribution helpers)
  meter: "compute.editor",
  quantity: (endMs - startMs) / 1000,
  unit: "seconds", // optional; checked against the catalog
  repoId, // optional; powers "usage by repo"
  attributes: { session: id }, // small JSON; stored and sent to Analytics Engine
  idempotencyKey: `editor:${sessionId}:${segment}`, // stable per logical event, max 200 chars
  at: endMs, // optional event time; decides the period
});

The call returns false when the key was already recorded, or when quantity is 0. It throws MeteringError on an unknown meter, a gauge meter, a wrong unit, a negative quantity or a bad key.

Idempotency keys. Prefix the key with your subsystem and derive it from durable ids, never from Date.now() or randomId(). Examples: ci:<jobId>, editor:<sessionId>:<heartbeatSeq>, integration:<deliveryId>, mcp:<requestId>, ai:<runId>:<callIndex>.

Long-running compute should record in segments: one event per heartbeat or per stop, each covering [lastRecordedAt, now] and keyed by the segment. This bills partial usage when a container crashes, and a replayed segment is a no-op.

recordUsageSoon(ctx, input)

Use this on hot request paths. It runs the same logic in ctx.waitUntil and logs failures instead of throwing, so metering never fails a user's request.

setGauge(accountId, meter, value)

For storage gauges your subsystem owns (storage.r2). The billed amount is the current level, and the period peak is kept.

Hard caps: reserveUsage, commitUsage and releaseUsage

const reservation = await reserveUsage({
  id: `editor:${sessionId}:${seq}`,
  accountId,
  meter: "compute.editor",
  quantity: 600,
  ttlMs: 15 * 60_000,
});
// throws QuotaExceededError (map it to 402 or 429) when the reservation won't fit
try {
  /* do the work */
  await commitUsage(reservation, { quantity: actualSeconds, repoId, attributes });
} catch (error) {
  await releaseUsage(reservation.id);
  throw error;
}
  • id is idempotent. Reserving the same id again returns the original reservation without holding more.
  • commitUsage releases the hold and records the real amount (key reservation:<id> unless you pass idempotencyKey) in one D1 batch. Calling it twice is safe.
  • If you never settle, cron releases the hold after ttlMs (default 1 hour), and nothing is billed.
  • assertWithinQuota(accountId, meter, quantity?) is the read-only pre-check. Use it to fail fast before expensive setup. It does not hold anything.

Soft meters, meaning those without a hard cap on the account's plan, always pass. They raise budget alerts instead.

Plans, caps and spending limits (src/lib/plans.ts)

  • The plans are scratch (anonymous), free and pro. Override them with the METERING_PLANS var (JSON in the same shape).
  • included is the monthly allowance. On free, it is a soft cap that sends alerts at 50, 80 and 100%. hardCaps block at the limit: on free these are AI calls and tokens and editor seconds; on scratch they are site storage, requests and egress, and AI and editor are 0.
  • On pro, overage is billed at microCentsPerUnit. A user-set spending limit turns each priced meter into a hard cap at included + limit / price. 0 means "never bill overage".
  • Budget alerts go to the user, or to every org owner, as notifications rows with reason billing and a link to the billing page. Each threshold fires once per period, deduped by usage_alerts.

Attribution

  • Usage is charged to the repo owner (repoOwnerAccount).
  • Fork PR CI is charged to the base repo owner. Fork runs execute in the base repository (workflow_runs.repo_id), and the base owner approves outside-contributor runs before cloud time is spent. ciRunAccount(runId) encodes this.
  • Agent usage (MCP or OAuth tokens) is charged to the account that authorized the agent (tokenAccount(tokenId) reads mcp_token_agents.authorized_by).
  • Anonymous scratch sites use scratchSiteAccount(siteId), and other anonymous usage uses scratchIpAccount(ipHash). On claim, claimScratchUsage(scratchId, accountId) merges the scratch aggregates into the claimer and redirects later events with that scratch id. Nothing is counted twice, because the rows move rather than being copied.
  • No double billing. Every meter has one recorder, and storage follows the LFS refcount precedent: shared objects are billed once per account. Forks start with a storage.artifacts estimate of 0, because Artifacts forks share objects.

Enforcement and known limits

  • Spending limits. withinSpendingLimit(accountId, meter?) sums the period's overage across every priced meter. Once the total reaches the limit, reserveUsage and assertWithinQuota refuse priced meters, and cloud CI dispatch holds new jobs as quota-exceeded. Other soft meters keep recording and alerting.
  • Streams. Git, raw, archive and asset bytes are counted as they stream. A client that disconnects is billed for the bytes it already received.
  • network.downloads is informational and unpriced. LFS content served through raw or archive downloads is charged once, on network.lfs. The same bytes also appear in network.downloads, so the money is billed once.
  • AI retries. aiMeter keys calls by <runId>:<index>. A replayed workflow step that calls the model again is not charged a second time. In exchange, retries can never double-bill.
  • CI. A job's runner seconds are recorded once, when it reaches its terminal state (key ci:<jobId>). If that D1 write fails, it is logged and not retried.
  • Claims. Once a scratch id is claimed it is bound to that account. A claim by any other account is rejected. In-flight holds move to the claimer together with the usage, and storage gauge rollups follow the claim.

HTTP

  • GET /api/v1/user/usage?period=YYYY-MM returns the viewer's summary.
  • GET /api/v1/orgs/:org/usage?period=YYYY-MM returns the org's summary (org members only).
  • The UI is at /settings/billing, /orgs/<org>/billing (owners can edit the limit) and /admin/billing (site admins).

The summary shape is UsageSummary in src/server/metering/summary.ts: { period, plan, spendingLimitCents, totalCents, meters[], repos[], days[], alerts[] }.

Integration status

Address: bisque-deer-9651 (branch roadmap/metering).

Area Status
Git proxy (ops, bytes, storage estimate) Wired: src/server/git/proxy.ts and metering/git.ts
LFS bandwidth Wired: chargeBandwidth in src/server/lfs/quota.ts
Raw, archive and asset downloads Wired: handleRawRoutes in src/server/raw.ts
CI runner seconds (cloud and self-hosted) and steps Wired: completeJob in src/server/ci/jobs.ts. The spending limit gates cloud dispatch in ci/dispatch.ts
AI (mergefix and Clef) Wired: aiMeter in metering/ai.ts, called from the planRun meter callback (mergefix owns that file from now on)
Storage gauges (LFS, CI cache, Artifacts estimate, sites, R2) Wired: cron rollupGauges
Sites requests, CPU, egress and scratch claim Hand-off: src/server/sites/metering.ts adapter (orangered-llama). At merge, replace its bodies with recordUsageSoon, assertWithinQuota and claimScratchUsage
Editor containers Wired: src/server/editor/container-meter.ts reserves the session budget on start and commits the exact seconds on every stop
Integration deliveries Hand-off: recordOutboundDelivery in src/server/integrations/metering.ts (lightblue-moth) becomes recordUsageSoon(... "ops.integration_deliveries" ..., key integration:<deliveryId>)
MCP tool calls Hand-off: recordToolUsage in src/server/mcp/usage.ts (slateblue-rabbit) becomes recordUsageSoon(... "ops.mcp_calls" ...)
Budget alerts Interim direct notifications insert in deliverAlerts. Switch to notifyQuotaThreshold from roadmap/notify at merge