# Price AI tokens per model, with margin per call

> Price AI calls per model and per input and output token, with a credit per model or a cost function. Bill line items that match your provider, and see each call's margin.

# Price AI tokens per model

What an AI call costs you depends on the model, and on how many tokens went in and came out. There are two ways to
meter that in a policy:

- **A credit per model and direction**, like `sonnet_input` and `sonnet_output`. Plain data, no code, and every
  credit is its own line item. Start here.
- **One credit with a cost function**: one metered unit for every model, with what each call cost you worked out by
  a small function in the policy.

Both record the real cost of every call next to what it's worth, so you know each call's margin as it happens. And
customers don't have to see either: one [AI credit](#sell-one-ai-credit) can pay for every model, through the
exchange, with no code.

## A credit per model

Each credit carries what you charge per token (`price`) and what the provider charges you (`overhead_cost`). After a
model call, record the usage the provider reported:

<Tabs groupId="lang" queryString>
<TabItem value="typescript" label="TypeScript">

```ts
const limitr = await Limitr.new(`
policy: {
    credits: {
        // Example rates per token: what you charge (price),
        // and what the provider charges you (overhead_cost).
        sonnet_input: {
            price: { amount: 0.000006 }     // $6 per million
            overhead_cost: 0.000003         // $3 per million
        }
        sonnet_output: {
            price: { amount: 0.00003 }      // $30 per million
            overhead_cost: 0.000015         // $15 per million
        }
        haiku_input: {
            price: { amount: 0.000002 }     // $2 per million
            overhead_cost: 0.000001         // $1 per million
        }
        haiku_output: {
            price: { amount: 0.00001 }      // $10 per million
            overhead_cost: 0.000005         // $5 per million
        }
    }
    plans: {
        pay_as_you_go: {
            default: true
            entitlements: {
                // Soft, with nothing included: every token is billed.
                sonnet_input: { limit: { credit: 'sonnet_input', mode: 'soft' } }
                sonnet_output: { limit: { credit: 'sonnet_output', mode: 'soft' } }
                haiku_input: { limit: { credit: 'haiku_input', mode: 'soft' } }
                haiku_output: { limit: { credit: 'haiku_output', mode: 'soft' } }
            }
        }
    }
}`);

// Each billed amount arrives as an event: one line item per credit.
limitr.addHandler('invoice', (name, value) => {
    if (name !== 'meter-overage') return;
    const { entitlement, overage, overage_price, meter } = JSON.parse(value as string);
    const cost = meter.overhead_diff;
    console.log(`${entitlement}: ${overage} tokens, bill $${overage_price}, cost $${cost}`);
});

await limitr.ensureCustomer('ada');

// After each model call, record the usage the provider reported.
type Usage = { input_tokens: number; output_tokens: number };

async function record(customer: string, model: string, usage: Usage) {
    await limitr.allow(customer, `${model}_input`, usage.input_tokens);
    await limitr.allow(customer, `${model}_output`, usage.output_tokens);
}

await record('ada', 'sonnet', { input_tokens: 2_000, output_tokens: 800 });
await record('ada', 'haiku', { input_tokens: 5_000, output_tokens: 1_200 });

const totals = await limitr.customerMarginSnapshot('ada');
const revenue = (totals?.get('revenue') as number).toFixed(3);
const cost = (totals?.get('cost') as number).toFixed(3);
console.log(`so far: revenue $${revenue}, cost $${cost}, margin ${totals?.get('margin')}%`);
```

It prints:

```text
sonnet_input: 2000 tokens, bill $0.012, cost $0.006
sonnet_output: 800 tokens, bill $0.024, cost $0.012
haiku_input: 5000 tokens, bill $0.01, cost $0.005
haiku_output: 1200 tokens, bill $0.012, cost $0.006
so far: revenue $0.058, cost $0.029, margin 50%
```

</TabItem>
<TabItem value="python" label="Python">

```python
from limitr import Limitr

limitr = Limitr("""
policy: {
    credits: {
        // Example rates per token: what you charge (price),
        // and what the provider charges you (overhead_cost).
        sonnet_input: {
            price: { amount: 0.000006 }     // $6 per million
            overhead_cost: 0.000003         // $3 per million
        }
        sonnet_output: {
            price: { amount: 0.00003 }      // $30 per million
            overhead_cost: 0.000015         // $15 per million
        }
        haiku_input: {
            price: { amount: 0.000002 }     // $2 per million
            overhead_cost: 0.000001         // $1 per million
        }
        haiku_output: {
            price: { amount: 0.00001 }      // $10 per million
            overhead_cost: 0.000005         // $5 per million
        }
    }
    plans: {
        pay_as_you_go: {
            default: true
            entitlements: {
                // Soft, with nothing included: every token is billed.
                sonnet_input: { limit: { credit: 'sonnet_input', mode: 'soft' } }
                sonnet_output: { limit: { credit: 'sonnet_output', mode: 'soft' } }
                haiku_input: { limit: { credit: 'haiku_input', mode: 'soft' } }
                haiku_output: { limit: { credit: 'haiku_output', mode: 'soft' } }
            }
        }
    }
}""")

# Each billed amount arrives as an event: one line item per credit.
def invoice(name, event):
    if name != 'meter-overage':
        return
    cost = event['meter']['overhead_diff']
    print(f"{event['entitlement']}: {event['overage']:g} tokens, bill ${event['overage_price']:g}, cost ${cost:g}")

limitr.add_handler('invoice', invoice)
limitr.ensure_customer('ada')

# After each model call, record the usage the provider reported.
def record(customer, model, usage):
    limitr.allow(customer, f'{model}_input', usage['input_tokens'])
    limitr.allow(customer, f'{model}_output', usage['output_tokens'])

record('ada', 'sonnet', {'input_tokens': 2_000, 'output_tokens': 800})
record('ada', 'haiku', {'input_tokens': 5_000, 'output_tokens': 1_200})

totals = limitr.customer_margin_snapshot('ada')
print(f"so far: revenue ${totals['revenue']:.3f}, cost ${totals['cost']:.3f}, margin {totals['margin']:g}%")
```

It prints:

```text
sonnet_input: 2000 tokens, bill $0.012, cost $0.006
sonnet_output: 800 tokens, bill $0.024, cost $0.012
haiku_input: 5000 tokens, bill $0.01, cost $0.005
haiku_output: 1200 tokens, bill $0.012, cost $0.006
so far: revenue $0.058, cost $0.029, margin 50%
```

</TabItem>
</Tabs>

- **Every credit is a line item.** Each `meter-overage` event names the entitlement, the tokens, and what to bill,
  so your invoice can show the same lines your provider's does: Sonnet input, Sonnet output, and so on.
- **Margin per model comes for free.** Each credit knows its own price and cost, and every event carries both.
- **`allow` decides and records in one pass.** Here the limits are soft with nothing included, so every call is
  allowed and billed. To gate calls as well, see [Patterns](#patterns).
- **A new model is two credits and two entitlements.** A provider price change is a change to `overhead_cost`, not
  a deploy.
- **No code in the policy**, so it works the same written in JSON, YAML, or TOML.

## Sell one AI credit

Customers often shouldn't have to think about models. Keep metering per model, and give them one abstract credit
that pays for all of them through the [exchange](../reference/exchange):

```stof
policy: {
    exchange: {
        ai_credit: { value: 0.01, currency: 'usd' }      // 1 AI credit is 1 cent
    }
    credits: {
        ai_credit: { label: 'AI credit' }                // what customers hold
        sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
        sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
        haiku_input: { price: { amount: 0.000002 }, overhead_cost: 0.000001 }
        haiku_output: { price: { amount: 0.00001 }, overhead_cost: 0.000005 }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                // Nothing included: AI credits pay for every token.
                sonnet_input: { limit: { credit: 'sonnet_input', value: 0 } }
                sonnet_output: { limit: { credit: 'sonnet_output', value: 0 } }
                haiku_input: { limit: { credit: 'haiku_input', value: 0 } }
                haiku_output: { limit: { credit: 'haiku_output', value: 0 } }
            }
            topups: {
                monthly_ai_credits: {
                    credit: 'ai_credit', value: 1_000, included: true
                    resets: true, reset_sch: 'monthly:1'
                }
                ai_credits_5000: { credit: 'ai_credit', value: 5_000, price: { amount: 45 } }
            }
        }
    }
}
```

- **Each model draws AI credits down at its own price.** At 1 cent a credit, 1,000 Sonnet output tokens cost 3
  credits and 1,000 Haiku output tokens cost 1.
- **Metering stays per model.** Every model keeps its own meter, price, cost, and events, so margin and usage per
  model work exactly as above. The AI credit is only what customers see, buy, and spend.
- **Hard limits at 0** mean calls run on credits until they run out. Make them `soft` to bill past the credits.
- **You can price the models in AI credits** instead of dollars, and reprice all of them by changing what an AI
  credit is worth. See [Pricing credits through the exchange](../reference/exchange#pricing-credits-through-the-exchange).

The [Exchange](../reference/exchange#discrete-and-abstract-credits) reference runs this policy end to end.

## A cost function on one credit

A cost function answers a different question: what one call cost you, when a single metered credit covers every
model. Use it when you meter one unit across models (one `chat` meter in `ai_token`), charge the same per token
whatever the model, and want the cost of each call to follow the model it used. You don't need one to sell a single
AI credit (above).

The rates live on the credit as data, and `fn overhead` prices each call from the event data you pass:

<Tabs groupId="lang" queryString>
<TabItem value="typescript" label="TypeScript">

```ts
const limitr = await Limitr.new(`
policy: {
    credits: {
        ai_token: {
            label: 'AI token'
            price: { amount: 0.00002 }          // you charge $20 per million tokens
            overhead_cost: 0.000005             // fallback cost per token
            rates: {                            // what each model costs you
                large: { input: 0.000003, output: 0.000015 }
                small: { input: 0.0000008, output: 0.000004 }
            }
            fn overhead(units: float, context?: obj) -> float {
                const rate = self.rates.get(context?.model ?? '');
                if (rate == null) return units * self.overhead_cost;
                const input = context.input ?? 0;
                const output = context.output ?? Num::max(units - input, 0);
                input * rate.input + output * rate.output
            }
        }
    }
    plans: {
        usage: {
            default: true
            entitlements: {
                chat: { limit: { credit: 'ai_token', mode: 'soft' } }
            }
        }
    }
}`);

limitr.addHandler('margin', (name, value) => {
    if (name !== 'meter-overage') return;
    const { event_data, meter } = JSON.parse(value as string);
    const charged = meter.price_diff.toFixed(4);
    const cost = meter.overhead_diff.toFixed(4);
    const margin = (meter.price_diff - meter.overhead_diff).toFixed(4);
    console.log(`${event_data.model}: charged $${charged}, cost $${cost}, margin $${margin}`);
});

await limitr.ensureCustomer('ada');

// The same 3,000-token conversation on two models, with what each provider reported.
await limitr.allow('ada', 'chat', 3000, { model: 'large', input: 2000, output: 1000 });
await limitr.allow('ada', 'chat', 3000, { model: 'small', input: 2000, output: 1000 });

// A model the policy has no rates for falls back to overhead_cost.
await limitr.allow('ada', 'chat', 3000, { model: 'new-model', input: 2000, output: 1000 });

// When the provider reports the exact cost, pass it and the engine records that instead.
await limitr.allow('ada', 'chat', 3000, { model: 'large', input: 2000, output: 1000 }, 0.0198);
```

It prints:

```text
large: charged $0.0600, cost $0.0210, margin $0.0390
small: charged $0.0600, cost $0.0056, margin $0.0544
new-model: charged $0.0600, cost $0.0150, margin $0.0450
large: charged $0.0600, cost $0.0198, margin $0.0402
```

</TabItem>
<TabItem value="python" label="Python">

```python
from limitr import Limitr

limitr = Limitr("""
policy: {
    credits: {
        ai_token: {
            label: 'AI token'
            price: { amount: 0.00002 }          // you charge $20 per million tokens
            overhead_cost: 0.000005             // fallback cost per token
            rates: {                            // what each model costs you
                large: { input: 0.000003, output: 0.000015 }
                small: { input: 0.0000008, output: 0.000004 }
            }
            fn overhead(units: float, context?: obj) -> float {
                const rate = self.rates.get(context?.model ?? '');
                if (rate == null) return units * self.overhead_cost;
                const input = context.input ?? 0;
                const output = context.output ?? Num::max(units - input, 0);
                input * rate.input + output * rate.output
            }
        }
    }
    plans: {
        usage: {
            default: true
            entitlements: {
                chat: { limit: { credit: 'ai_token', mode: 'soft' } }
            }
        }
    }
}""")

def margin(name, event):
    if name != 'meter-overage':
        return
    meter = event['meter']
    charged, cost = meter['price_diff'], meter['overhead_diff']
    print(f"{event['event_data']['model']}: charged ${charged:.4f}, cost ${cost:.4f}, margin ${charged - cost:.4f}")

limitr.add_handler('margin', margin)
limitr.ensure_customer('ada')

# The same 3,000-token conversation on two models, with what each provider reported.
limitr.allow('ada', 'chat', 3000, {'model': 'large', 'input': 2000, 'output': 1000})
limitr.allow('ada', 'chat', 3000, {'model': 'small', 'input': 2000, 'output': 1000})

# A model the policy has no rates for falls back to overhead_cost.
limitr.allow('ada', 'chat', 3000, {'model': 'new-model', 'input': 2000, 'output': 1000})

# When the provider reports the exact cost, pass it and the engine records that instead.
limitr.allow('ada', 'chat', 3000, {'model': 'large', 'input': 2000, 'output': 1000}, overhead=0.0198)
```

It prints:

```text
large: charged $0.0600, cost $0.0210, margin $0.0390
small: charged $0.0600, cost $0.0056, margin $0.0544
new-model: charged $0.0600, cost $0.0150, margin $0.0450
large: charged $0.0600, cost $0.0198, margin $0.0402
```

</TabItem>
</Tabs>

- **`fn overhead(units, context?)`** prices one call. `context` is the event data passed to `allow` (or `settle`):
  here the model and the provider's input and output token counts.
- **Default every field with `??`.** `context` can be null, a model can be missing from `rates`, and before a call
  runs (`reserve`) there's no `output` yet. Arithmetic with null is an error in Stof.
- **If it throws**, the engine logs the error and uses `units * overhead_cost`, so a mistake never breaks a request.
- **Keep it pure**: read the policy, do the math, return a number.
- **The exact cost wins.** When the provider reports a call's cost, pass it as `overhead` (the fifth argument to
  `allow`, or to `settle`) and the engine records that instead of running the function.

## Which to use

| | A credit per model | Per model, sold as AI credits | A cost function |
|---|---|---|---|
| What you meter | Tokens per model and direction | Tokens per model and direction | One unit across models |
| What customers see | Tokens per model | One AI credit balance | One unit (Ex. AI tokens) |
| Price per token | Per model | Per model, paid in AI credits | The same for every model |
| Invoice lines | Per model, matching the provider | Credit packs, plus usage per model | One for the unit |
| Policy format | Any: JSON, YAML, TOML, or Stof | Any | Stof, for the function |

These also mix: a credit per model for what you bill, and one entitlement in your own unit for control (below).

## Patterns

Once the basics are in place, these are the setups that come up most.

### Gate before, bill after

Check that a customer has room before the call, and record the provider's numbers after. With a hard limit, the call
that crosses it still has to be recorded, because it already happened: pass `force` (the sixth argument to `allow`),
and the next `check` is denied. That's a trailing hard limit.

<Tabs groupId="lang" queryString>
<TabItem value="typescript" label="TypeScript">

```ts norun
if (!(await limitr.check(userId, 'sonnet_output', maxTokens))) return tooManyTokens();

const usage = await callModel(model, messages, maxTokens);
await limitr.allow(userId, 'sonnet_input', usage.input_tokens, true, undefined, true);
await limitr.allow(userId, 'sonnet_output', usage.output_tokens, true, undefined, true);
```

</TabItem>
<TabItem value="python" label="Python">

```python norun
if not limitr.check(user_id, 'sonnet_output', max_tokens):
    return too_many_tokens()

usage = call_model(model, messages, max_tokens)
limitr.allow(user_id, 'sonnet_input', usage.input_tokens, force=True)
limitr.allow(user_id, 'sonnet_output', usage.output_tokens, force=True)
```

</TabItem>
</Tabs>

### Control in your own count, bill in the provider's

Providers count tokens differently, and you may switch providers over time. Keep control in a unit of your own (your
count of the prompt, plus the most the answer can use), and billing in per-model credits filled with what the
provider actually charged for:

<Tabs groupId="lang" queryString>
<TabItem value="typescript" label="TypeScript">

```ts
const limitr = await Limitr.new(`
policy: {
    credits: {
        token: { label: 'Token' }           // your own count, the same for every model
        sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
        sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                // Control: 20,000 tokens a day by your count, whatever the model.
                chat: {
                    limit: {
                        credit: 'token', mode: 'hard', value: 20_000
                        resets: true, reset_inc: 1day
                    }
                }

                // Billing: exactly what the provider reports, every token billed.
                sonnet_input: { limit: { credit: 'sonnet_input', mode: 'soft' } }
                sonnet_output: { limit: { credit: 'sonnet_output', mode: 'soft' } }
            }
        }
    }
}`);

await limitr.ensureCustomer('ada');

// Stand-ins for your tokenizer and your model provider.
const countTokens = (prompt: string) => Math.ceil(prompt.length / 4);
const callModel = async (prompt: string, maxTokens: number) => ({
    // The provider counts a little differently
    input_tokens: countTokens(prompt) + 12,
    output_tokens: Math.min(maxTokens, 700),
});

async function chat(customer: string, prompt: string, maxTokens = 1_000) {
    // 1. Control: your count of the prompt, plus the most the answer can use.
    const tokens = countTokens(prompt) + maxTokens;
    if (!(await limitr.allow(customer, 'chat', tokens))) return 'denied';

    // 2. The call.
    const usage = await callModel(prompt, maxTokens);

    // 3. Billing: what the provider actually charged for.
    await limitr.allow(customer, 'sonnet_input', usage.input_tokens);
    await limitr.allow(customer, 'sonnet_output', usage.output_tokens);
    return `billed ${usage.input_tokens} in, ${usage.output_tokens} out`;
}

const prompt = 'Summarize this contract. '.repeat(400);    // about 2,500 tokens
for (let i = 1; i <= 6; i++) console.log(`call ${i}:`, await chat('ada', prompt));
console.log('control used today:', await limitr.value('ada', 'chat'));
```

It prints:

```text
call 1: billed 2512 in, 700 out
call 2: billed 2512 in, 700 out
call 3: billed 2512 in, 700 out
call 4: billed 2512 in, 700 out
call 5: billed 2512 in, 700 out
call 6: denied
control used today: 17500
```

</TabItem>
<TabItem value="python" label="Python">

```python
from limitr import Limitr

limitr = Limitr("""
policy: {
    credits: {
        token: { label: 'Token' }           // your own count, the same for every model
        sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
        sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                // Control: 20,000 tokens a day by your count, whatever the model.
                chat: {
                    limit: {
                        credit: 'token', mode: 'hard', value: 20_000
                        resets: true, reset_inc: 1day
                    }
                }

                // Billing: exactly what the provider reports, every token billed.
                sonnet_input: { limit: { credit: 'sonnet_input', mode: 'soft' } }
                sonnet_output: { limit: { credit: 'sonnet_output', mode: 'soft' } }
            }
        }
    }
}""")

limitr.ensure_customer('ada')

# Stand-ins for your tokenizer and your model provider.
def count_tokens(prompt):
    return math.ceil(len(prompt) / 4)

def call_model(prompt, max_tokens):
    # The provider counts a little differently
    return {'input_tokens': count_tokens(prompt) + 12, 'output_tokens': min(max_tokens, 700)}

def chat(customer, prompt, max_tokens=1_000):
    # 1. Control: your count of the prompt, plus the most the answer can use.
    tokens = count_tokens(prompt) + max_tokens
    if not limitr.allow(customer, 'chat', tokens):
        return 'denied'

    # 2. The call.
    usage = call_model(prompt, max_tokens)

    # 3. Billing: what the provider actually charged for.
    limitr.allow(customer, 'sonnet_input', usage['input_tokens'])
    limitr.allow(customer, 'sonnet_output', usage['output_tokens'])
    return f"billed {usage['input_tokens']} in, {usage['output_tokens']} out"

prompt = 'Summarize this contract. ' * 400    # about 2,500 tokens
for i in range(1, 7):
    print(f'call {i}:', chat('ada', prompt))
print('control used today:', f"{limitr.value('ada', 'chat'):g}")
```

It prints:

```text
call 1: billed 2512 in, 700 out
call 2: billed 2512 in, 700 out
call 3: billed 2512 in, 700 out
call 4: billed 2512 in, 700 out
call 5: billed 2512 in, 700 out
call 6: denied
control used today: 17500
```

</TabItem>
</Tabs>

- **Control stays the same** when you add a model or change providers: `chat` only knows your count.
- **Billing stays exact**: the per-model entitlements only ever see the provider's numbers.
- **It's more credits and entitlements**, which is what a policy is for. Each one is a few lines.
- **To count what the answer really used** instead of the most it could, [reserve and settle](./reserve-llm-calls)
  `chat` instead of allowing it up front.

### One budget across models

Per-model limits count tokens per model. To stop a customer at a dollar amount across all of them, add a
[spend cap](../reference/spend-caps). Caps count what each call is worth at your price, across every credit:

<Tabs groupId="lang" queryString>
<TabItem value="typescript" label="TypeScript">

```ts norun
await limitr.addCustomerCap(userId, 50, {
    cap_id: 'ai_budget',
    resets: true, reset_sch: 'monthly:1',
});
```

</TabItem>
<TabItem value="python" label="Python">

```python norun
limitr.add_customer_cap(user_id, 50, cap_id='ai_budget', resets=True, reset_sch='monthly:1')
```

</TabItem>
</Tabs>

With `hard_trailing: true`, the call that crosses $50 goes through and the calls after it are denied, which suits
usage recorded after the call.

### Cap what a customer can cost you

A cap with `overhead_cost: true` counts cost instead of price, so a customer can't cost you more than a set amount,
whatever they're charged:

<Tabs groupId="lang" queryString>
<TabItem value="typescript" label="TypeScript">

```ts norun
await limitr.addCustomerCap(userId, 20, {
    cap_id: 'cost_guard',
    // Count what calls cost you, not what you charge
    overhead_cost: true,
    resets: true, reset_sch: 'monthly:1',
});
```

</TabItem>
<TabItem value="python" label="Python">

```python norun
limitr.add_customer_cap(
    user_id, 20,
    cap_id='cost_guard',
    overhead_cost=True,     # count what calls cost you, not what you charge
    resets=True, reset_sch='monthly:1',
)
```

</TabItem>
</Tabs>

## Keeping rates current

Rates are data in the policy, whichever way you price, so a provider price change is a policy change, not a deploy:

- Self-hosted: change the policy file and ship it like any other config change. Customer state lives in your
  database, so it carries over.
- [Limitr Cloud](../tutorial/going-to-cloud): edit the rates in Cloud and publish; every running engine gets them
  within seconds.
- From your app at runtime: when models are priced through the exchange, a scheduled job can reprice one with
  `setExchangePair` (Ex. `setExchangePair('sonnet_output', 0.0035, 'ai_credit')`). See
  [Live rates](../reference/exchange#live-rates).

The cost of each call is recorded on its meter when it happens, so a rate change never re-prices past usage.

## Next

- Calls whose size you only know afterwards: [Reserve and settle LLM calls](./reserve-llm-calls).
- Control next to billing, in general: [Rate limits next to paid usage](./rate-limits).
- One cost function for several credits: [Cost functions](../reference/cost-and-margin#one-cost-function-for-many-credits).
