Per-customer cost attribution without sending us a single prompt

Northbeam Health builds intake and prior-authorisation automation for mid-sized clinical networks. Their agents read referral documents and payer policies, both of which are patient-adjacent, and their business associate agreements are unambiguous: that text does not leave their infrastructure.

That constraint had, until last year, been treated as a reason they could not have observability at all.

Structure without content

Cinder’s payload capture is opt-in per attribute, and Northbeam turned all of it off. What they send is span shape: which step ran, how long it took, which model was called, how many tokens in and out, which tenant the trace belongs to, and whether validation passed.

“People assume the prompt is the interesting part,” says Fola Abara, Engineering Manager. “For operational questions it almost never is. I want to know that tenant 41 is spending nine times the median per case and why. That is all timing and token counts.”

The tenant ID is attached at the root span and propagates, so every child span inherits it without any per-call plumbing. Cost is derived from token counts and the model identifier — including Bedrock cross-region inference profiles, which resolve to the model that actually served the request rather than the profile that was requested.

What the attribution found

Three things, in the first fortnight.

A single tenant with unusually long referral documents was driving 22% of total inference spend against 4% of case volume, because their documents were being passed whole rather than chunked. That tenant now goes through a summarisation pass first.

A retry policy was double-charging on timeouts: the original request had usually completed server-side and been billed before the client gave up. Adding an idempotency key and extending the client timeout removed most of it.

And a routing rule that was supposed to send simple cases to a smaller model had been inverted since a refactor in February. It had cost roughly $18,000 and had produced no observable symptom, because both models answered correctly.

Together those took inference spend down 31% in a quarter, on flat case volume.

The billing side effect

Northbeam bills their clinical networks partly on usage. That reconciliation had been a monthly spreadsheet assembled by hand from CloudWatch exports, and it was the thing everyone dreaded.

The per-tenant cost breakdown replaced it in two days, using a scheduled export to their own S3 bucket rather than anything bespoke. “It is not what we bought the product for,” Abara says. “It is the thing finance mentions.”