Cost functions and margin

Every metered call has two numbers: what it's worth at your list price, and what it cost you. Limitr records both, so margin is known per call, as it happens, instead of at the end of the month.

What a call costs you

The engine prices a call's cost (its overhead, in runes) in this order:

1. The actual cost, when you pass it: allow(id, entitlement, value, event, overhead) or settle(..., overhead). Use it when the provider reports the cost of a call.
2. The credit's cost function, fn overhead(units, context?) -> float, when the policy defines one.
3. The fixed rate: units * overhead_cost.

A fixed rate per credit

Most policies need nothing more than overhead_cost. For AI usage, give each model and direction its own credit, so each has its own rate and becomes its own line item:

credits: {
    sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
    sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
}

See AI costs per model for when to use per-model credits and when to use a cost function.

Cost functions

A cost function prices one call from the call's own data. context is the event data you pass with the call, so a function can price input and output tokens per model:

policy: {
    credits: {
        ai_token: {
            label: 'AI token'
            price: { amount: 0.00002 }
            overhead_cost: 0.000005                    // fallback cost per token
            rates: {                                   // what each model costs you
                large: { input: 0.000003, output: 0.000015 }
                small: { input: 0.000001, output: 0.000005 }
            }
            fn overhead(units: float, context?: obj) -> float {
                const rate = self.rates.get(context?.model ?? '');
                if (rate == null) return units * self.overhead_cost;
                const input = context.input ?? 0;
                // Before the call there's no output yet: the rest is output.
                const output = context.output ?? Num::max(units - input, 0);
                input * rate.input + output * rate.output
            }
        }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                chat: {
                    limit: {
                        credit: 'ai_token', mode: 'soft', value: 2_000_000
                        resets: true
                    }
                }
            }
        }
    }
}

await limitr.allow(userId, 'chat', 1500, { model: 'large', input: 1200, output: 300 });

• The rates are data. Change them in the policy when a provider changes its prices, without a deploy. With Limitr Cloud, a policy change reaches every running engine.
• Default every field. context can be null, and fields can be missing (reserve prices a hold before the call has an output). Arithmetic with null is an error in Stof, so use ?? for every field you read.
• Failures are safe. If the function throws or returns something other than a number, the engine logs the error (console.error in JavaScript) and uses units * overhead_cost.
• Keep it pure. Cost functions run on every call, including check. Read the policy, do math, return a number. Don't change the policy or create objects.

One cost function for many credits

Several credits can share one function: point overhead at a function in a #[type]. A shared function isn't defined on the credit, so self inside it is not the credit. Keep what it reads next to it, on the type:

#[type]
Costs: {
    rates: {
        large: { input: 0.000003, output: 0.000015 }
        small: { input: 0.000001, output: 0.000005 }
    }

    #[static]
    fn per_model(units: float, context?: obj) -> float {
        const rate = <Costs>.rates.get(context?.model ?? '');
        if (rate == null) return units * 0.000005;
        const input = context.input ?? 0;
        const output = context.output ?? Num::max(units - input, 0);
        input * rate.input + output * rate.output
    }
}

policy: {
    credits: {
        chat_token: {
            label: 'Chat token'
            price: { amount: 0.00002 }
            overhead: <Costs>.per_model
        }
        code_token: {
            label: 'Code token'
            price: { amount: 0.00003 }
            overhead: <Costs>.per_model
        }
    }
}

Cost with no usage

A call that failed can still cost you. Record the cost alone with a value of 0:

// $0.0042 of cost, no tokens
await limitr.allow(userId, 'chat', 0, { model: 'large' }, 0.0042);

It sends a meter-changed event and counts toward overhead_cost caps.

What a call is worth

A call's list price is what its billable units are worth: the units past the plan's limit, priced by the credit's pricing model from where the customer already is this period. Units inside the allowance are paid for by the plan, so they have cost but no list price.

Margin in events

Every meter event carries both numbers for the call, in runes:

Field	Meaning
meter.overhead_diff	What this call cost you.
meter.overhead	Cost recorded this period, so far.
meter.price_diff	What this call is worth at list price, before grants. Null when the credit has no price.
overage_price	On meter-overage: what to bill, after grants covered what they could.

price_diff - overhead_diff is the call's margin:

limitr.addHandler('margin', (name, value) => {
    if (name !== 'meter-changed' && name !== 'meter-overage') return;
    const { customer, entitlement, meter } = JSON.parse(value as string);
    const margin = (meter.price_diff ?? 0) - meter.overhead_diff;
    analytics.track('margin', { customer: customer.id, entitlement, margin });
});

Prices are only computed when a call sends events (event isn't false).

Cost is recorded, not recalculated

Each call's cost is added to its meter's overhead for the period. Margin reads use that record, so a price change in the middle of a month never re-prices the usage that came before it. The record resets with the meter.

Margin snapshots

One customer, this period. customerMarginSnapshot(customer) (customer_local_margin_breakdown in Stof) returns revenue (list price of billable usage), cost (recorded overhead), and margin as a percent, overall and per entitlement:

Map(4) {
  'cost' => 0.12,
  'entitlements' => Map(1) { 'chat' => Map(3) { 'cost' => 0.12, 'margin' => -20, 'revenue' => 0.1 } },
  'margin' => -20,
  'revenue' => 0.1
}

It's a quick check, not a ledger: it doesn't count plan fees, grants, or topups. A customer with costs and no billable usage shows a margin of -100.

A plan, before anyone uses it. marginSnapshot(plan, usage) (local_margin_breakdown) prices hypothetical usage: a map of entitlement to value, or to a map with value, overhead, or context (passed to the cost function). Use it to check a plan's margin while designing it.

const projected = await limitr.marginSnapshot('pro', new Map([['chat', 50_000]]));

A session or pipeline. In TypeScript, startMarginMeasurement(customer, id) adds two observe-only caps (one for what was charged, one for cost), and captureMarginMeasurement(customer, id) returns { charged, costs, margin, rawMargin } and removes them. Use it to measure one job or agent run.

Capping cost

A cap with overhead_cost: true limits what a customer can cost you, whatever they're charged:

await limitr.addCustomerCap(userId, 20, {
    cap_id: 'cost_guard',
    // Count what calls cost you, not what you charge
    overhead_cost: true,
    resets: true, reset_sch: 'monthly:1',
});

See Spend caps.