Price AI tokens per model

What an AI call costs you depends on the model, and on how many tokens went in and came out. There are two ways to meter that in a policy:

• A credit per model and direction, like sonnet_input and sonnet_output. Plain data, no code, and every credit is its own line item. Start here.
• One credit with a cost function: one metered unit for every model, with what each call cost you worked out by a small function in the policy.

Both record the real cost of every call next to what it's worth, so you know each call's margin as it happens. And customers don't have to see either: one AI credit can pay for every model, through the exchange, with no code.

A credit per model

Each credit carries what you charge per token (price) and what the provider charges you (overhead_cost). After a model call, record the usage the provider reported:

import { Limitr } from '@formata/limitr';

const limitr = await Limitr.new(`
policy: {
    credits: {
        // Example rates per token: what you charge (price),
        // and what the provider charges you (overhead_cost).
        sonnet_input: {
            price: { amount: 0.000006 }     // $6 per million
            overhead_cost: 0.000003         // $3 per million
        }
        sonnet_output: {
            price: { amount: 0.00003 }      // $30 per million
            overhead_cost: 0.000015         // $15 per million
        }
        haiku_input: {
            price: { amount: 0.000002 }     // $2 per million
            overhead_cost: 0.000001         // $1 per million
        }
        haiku_output: {
            price: { amount: 0.00001 }      // $10 per million
            overhead_cost: 0.000005         // $5 per million
        }
    }
    plans: {
        pay_as_you_go: {
            default: true
            entitlements: {
                // Soft, with nothing included: every token is billed.
                sonnet_input: { limit: { credit: 'sonnet_input', mode: 'soft' } }
                sonnet_output: { limit: { credit: 'sonnet_output', mode: 'soft' } }
                haiku_input: { limit: { credit: 'haiku_input', mode: 'soft' } }
                haiku_output: { limit: { credit: 'haiku_output', mode: 'soft' } }
            }
        }
    }
}`);

// Each billed amount arrives as an event: one line item per credit.
limitr.addHandler('invoice', (name, value) => {
    if (name !== 'meter-overage') return;
    const { entitlement, overage, overage_price, meter } = JSON.parse(value as string);
    const cost = meter.overhead_diff;
    console.log(`${entitlement}: ${overage} tokens, bill $${overage_price}, cost $${cost}`);
});

await limitr.ensureCustomer('ada');

// After each model call, record the usage the provider reported.
type Usage = { input_tokens: number; output_tokens: number };

async function record(customer: string, model: string, usage: Usage) {
    await limitr.allow(customer, `${model}_input`, usage.input_tokens);
    await limitr.allow(customer, `${model}_output`, usage.output_tokens);
}

await record('ada', 'sonnet', { input_tokens: 2_000, output_tokens: 800 });
await record('ada', 'haiku', { input_tokens: 5_000, output_tokens: 1_200 });

const totals = await limitr.customerMarginSnapshot('ada');
const revenue = (totals?.get('revenue') as number).toFixed(3);
const cost = (totals?.get('cost') as number).toFixed(3);
console.log(`so far: revenue $${revenue}, cost $${cost}, margin ${totals?.get('margin')}%`);

It prints:

sonnet_input: 2000 tokens, bill $0.012, cost $0.006
sonnet_output: 800 tokens, bill $0.024, cost $0.012
haiku_input: 5000 tokens, bill $0.01, cost $0.005
haiku_output: 1200 tokens, bill $0.012, cost $0.006
so far: revenue $0.058, cost $0.029, margin 50%

• Every credit is a line item. Each meter-overage event names the entitlement, the tokens, and what to bill, so your invoice can show the same lines your provider's does: Sonnet input, Sonnet output, and so on.
• Margin per model comes for free. Each credit knows its own price and cost, and every event carries both.
• allow decides and records in one pass. Here the limits are soft with nothing included, so every call is allowed and billed. To gate calls as well, see Patterns.
• A new model is two credits and two entitlements. A provider price change is a change to overhead_cost, not a deploy.
• No code in the policy, so it works the same written in JSON, YAML, or TOML.

Sell one AI credit

Customers often shouldn't have to think about models. Keep metering per model, and give them one abstract credit that pays for all of them through the exchange:

policy: {
    exchange: {
        ai_credit: { value: 0.01, currency: 'usd' }      // 1 AI credit is 1 cent
    }
    credits: {
        ai_credit: { label: 'AI credit' }                // what customers hold
        sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
        sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
        haiku_input: { price: { amount: 0.000002 }, overhead_cost: 0.000001 }
        haiku_output: { price: { amount: 0.00001 }, overhead_cost: 0.000005 }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                // Nothing included: AI credits pay for every token.
                sonnet_input: { limit: { credit: 'sonnet_input', value: 0 } }
                sonnet_output: { limit: { credit: 'sonnet_output', value: 0 } }
                haiku_input: { limit: { credit: 'haiku_input', value: 0 } }
                haiku_output: { limit: { credit: 'haiku_output', value: 0 } }
            }
            topups: {
                monthly_ai_credits: {
                    credit: 'ai_credit', value: 1_000, included: true
                    resets: true, reset_sch: 'monthly:1'
                }
                ai_credits_5000: { credit: 'ai_credit', value: 5_000, price: { amount: 45 } }
            }
        }
    }
}

• Each model draws AI credits down at its own price. At 1 cent a credit, 1,000 Sonnet output tokens cost 3 credits and 1,000 Haiku output tokens cost 1.
• Metering stays per model. Every model keeps its own meter, price, cost, and events, so margin and usage per model work exactly as above. The AI credit is only what customers see, buy, and spend.
• Hard limits at 0 mean calls run on credits until they run out. Make them soft to bill past the credits.
• You can price the models in AI credits instead of dollars, and reprice all of them by changing what an AI credit is worth. See Pricing credits through the exchange.

The Exchange reference runs this policy end to end.

A cost function on one credit

A cost function answers a different question: what one call cost you, when a single metered credit covers every model. Use it when you meter one unit across models (one chat meter in ai_token), charge the same per token whatever the model, and want the cost of each call to follow the model it used. You don't need one to sell a single AI credit (above).

The rates live on the credit as data, and fn overhead prices each call from the event data you pass:

import { Limitr } from '@formata/limitr';

const limitr = await Limitr.new(`
policy: {
    credits: {
        ai_token: {
            label: 'AI token'
            price: { amount: 0.00002 }          // you charge $20 per million tokens
            overhead_cost: 0.000005             // fallback cost per token
            rates: {                            // what each model costs you
                large: { input: 0.000003, output: 0.000015 }
                small: { input: 0.0000008, output: 0.000004 }
            }
            fn overhead(units: float, context?: obj) -> float {
                const rate = self.rates.get(context?.model ?? '');
                if (rate == null) return units * self.overhead_cost;
                const input = context.input ?? 0;
                const output = context.output ?? Num::max(units - input, 0);
                input * rate.input + output * rate.output
            }
        }
    }
    plans: {
        usage: {
            default: true
            entitlements: {
                chat: { limit: { credit: 'ai_token', mode: 'soft' } }
            }
        }
    }
}`);

limitr.addHandler('margin', (name, value) => {
    if (name !== 'meter-overage') return;
    const { event_data, meter } = JSON.parse(value as string);
    const charged = meter.price_diff.toFixed(4);
    const cost = meter.overhead_diff.toFixed(4);
    const margin = (meter.price_diff - meter.overhead_diff).toFixed(4);
    console.log(`${event_data.model}: charged $${charged}, cost $${cost}, margin $${margin}`);
});

await limitr.ensureCustomer('ada');

// The same 3,000-token conversation on two models, with what each provider reported.
await limitr.allow('ada', 'chat', 3000, { model: 'large', input: 2000, output: 1000 });
await limitr.allow('ada', 'chat', 3000, { model: 'small', input: 2000, output: 1000 });

// A model the policy has no rates for falls back to overhead_cost.
await limitr.allow('ada', 'chat', 3000, { model: 'new-model', input: 2000, output: 1000 });

// When the provider reports the exact cost, pass it and the engine records that instead.
await limitr.allow('ada', 'chat', 3000, { model: 'large', input: 2000, output: 1000 }, 0.0198);

It prints:

large: charged $0.0600, cost $0.0210, margin $0.0390
small: charged $0.0600, cost $0.0056, margin $0.0544
new-model: charged $0.0600, cost $0.0150, margin $0.0450
large: charged $0.0600, cost $0.0198, margin $0.0402

• fn overhead(units, context?) prices one call. context is the event data passed to allow (or settle): here the model and the provider's input and output token counts.
• Default every field with ??. context can be null, a model can be missing from rates, and before a call runs (reserve) there's no output yet. Arithmetic with null is an error in Stof.
• If it throws, the engine logs the error and uses units * overhead_cost, so a mistake never breaks a request.
• Keep it pure: read the policy, do the math, return a number.
• The exact cost wins. When the provider reports a call's cost, pass it as overhead (the fifth argument to allow, or to settle) and the engine records that instead of running the function.

Which to use

	A credit per model	Per model, sold as AI credits	A cost function
What you meter	Tokens per model and direction	Tokens per model and direction	One unit across models
What customers see	Tokens per model	One AI credit balance	One unit (Ex. AI tokens)
Price per token	Per model	Per model, paid in AI credits	The same for every model
Invoice lines	Per model, matching the provider	Credit packs, plus usage per model	One for the unit
Policy format	Any: JSON, YAML, TOML, or Stof	Any	Stof, for the function

These also mix: a credit per model for what you bill, and one entitlement in your own unit for control (below).

Patterns

Once the basics are in place, these are the setups that come up most.

Gate before, bill after

Check that a customer has room before the call, and record the provider's numbers after. With a hard limit, the call that crosses it still has to be recorded, because it already happened: pass force (the sixth argument to allow), and the next check is denied. That's a trailing hard limit.

if (!(await limitr.check(userId, 'sonnet_output', maxTokens))) return tooManyTokens();

const usage = await callModel(model, messages, maxTokens);
await limitr.allow(userId, 'sonnet_input', usage.input_tokens, true, undefined, true);
await limitr.allow(userId, 'sonnet_output', usage.output_tokens, true, undefined, true);

Control in your own count, bill in the provider's

Providers count tokens differently, and you may switch providers over time. Keep control in a unit of your own (your count of the prompt, plus the most the answer can use), and billing in per-model credits filled with what the provider actually charged for:

import { Limitr } from '@formata/limitr';

const limitr = await Limitr.new(`
policy: {
    credits: {
        token: { label: 'Token' }           // your own count, the same for every model
        sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
        sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                // Control: 20,000 tokens a day by your count, whatever the model.
                chat: {
                    limit: {
                        credit: 'token', mode: 'hard', value: 20_000
                        resets: true, reset_inc: 1day
                    }
                }

                // Billing: exactly what the provider reports, every token billed.
                sonnet_input: { limit: { credit: 'sonnet_input', mode: 'soft' } }
                sonnet_output: { limit: { credit: 'sonnet_output', mode: 'soft' } }
            }
        }
    }
}`);

await limitr.ensureCustomer('ada');

// Stand-ins for your tokenizer and your model provider.
const countTokens = (prompt: string) => Math.ceil(prompt.length / 4);
const callModel = async (prompt: string, maxTokens: number) => ({
    // The provider counts a little differently
    input_tokens: countTokens(prompt) + 12,
    output_tokens: Math.min(maxTokens, 700),
});

async function chat(customer: string, prompt: string, maxTokens = 1_000) {
    // 1. Control: your count of the prompt, plus the most the answer can use.
    const tokens = countTokens(prompt) + maxTokens;
    if (!(await limitr.allow(customer, 'chat', tokens))) return 'denied';

    // 2. The call.
    const usage = await callModel(prompt, maxTokens);

    // 3. Billing: what the provider actually charged for.
    await limitr.allow(customer, 'sonnet_input', usage.input_tokens);
    await limitr.allow(customer, 'sonnet_output', usage.output_tokens);
    return `billed ${usage.input_tokens} in, ${usage.output_tokens} out`;
}

const prompt = 'Summarize this contract. '.repeat(400);    // about 2,500 tokens
for (let i = 1; i <= 6; i++) console.log(`call ${i}:`, await chat('ada', prompt));
console.log('control used today:', await limitr.value('ada', 'chat'));

It prints:

call 1: billed 2512 in, 700 out
call 2: billed 2512 in, 700 out
call 3: billed 2512 in, 700 out
call 4: billed 2512 in, 700 out
call 5: billed 2512 in, 700 out
call 6: denied
control used today: 17500

• Control stays the same when you add a model or change providers: chat only knows your count.
• Billing stays exact: the per-model entitlements only ever see the provider's numbers.
• It's more credits and entitlements, which is what a policy is for. Each one is a few lines.
• To count what the answer really used instead of the most it could, reserve and settle chat instead of allowing it up front.

One budget across models

Per-model limits count tokens per model. To stop a customer at a dollar amount across all of them, add a spend cap. Caps count what each call is worth at your price, across every credit:

await limitr.addCustomerCap(userId, 50, {
    cap_id: 'ai_budget',
    resets: true, reset_sch: 'monthly:1',
});

With hard_trailing: true, the call that crosses $50 goes through and the calls after it are denied, which suits usage recorded after the call.

Cap what a customer can cost you

A cap with overhead_cost: true counts cost instead of price, so a customer can't cost you more than a set amount, whatever they're charged:

await limitr.addCustomerCap(userId, 20, {
    cap_id: 'cost_guard',
    // Count what calls cost you, not what you charge
    overhead_cost: true,
    resets: true, reset_sch: 'monthly:1',
});

Keeping rates current

Rates are data in the policy, whichever way you price, so a provider price change is a policy change, not a deploy:

• Self-hosted: change the policy file and ship it like any other config change. Customer state lives in your database, so it carries over.
• Limitr Cloud: edit the rates in Cloud and publish; every running engine gets them within seconds.
• From your app at runtime: when models are priced through the exchange, a scheduled job can reprice one with setExchangePair (Ex. setExchangePair('sonnet_output', 0.0035, 'ai_credit')). See Live rates.

The cost of each call is recorded on its meter when it happens, so a rate change never re-prices past usage.

Next

• Calls whose size you only know afterwards: Reserve and settle LLM calls.
• Control next to billing, in general: Rate limits next to paid usage.
• One cost function for several credits: Cost functions.