Exchange

The exchange converts any credit into any other. Every value resolves to runes, Limitr's base unit (one rune is one US dollar by default), so anything that resolves to runes converts into anything else that does. Currencies are credits too: rune and usd are built in, and you add others the same way.

That's what lets:

• an abstract credit pay for discrete ones: customers hold "AI credits", and each model's tokens draw them down at that model's own price;
• prices live in one place: price models in AI credits, and reprice all of them with one number;
• budgets and prices use any currency: a spend cap in euros on usage priced in dollars.

Discrete and abstract credits

A credit is just a credit, but for our mental model and strategies, it helps sometimes to define their configurations.

Discrete credits have an explicit definition tied to real value: each model's input and output tokens, images, seconds of video. Each has its own price and its own cost, so every call is priced and every model is its own line item.

Abstract credits are less defined, often applying to many features and rates at once: one balance, like "AI credits", that spends across everything. An abstract credit is a credit with no price of its own; its value comes from an exchange pair. Grants and topups in it pay for any discrete credit that converts into it:

import { Limitr } from '@formata/limitr';

const limitr = await Limitr.new(`
policy: {
    exchange: {
        // The credit customers hold: 1 AI credit is 1 cent.
        ai_credit: { value: 0.01, currency: 'usd' }
    }
    credits: {
        // What customers hold. Its value comes from the exchange, so it has no price.
        ai_credit: { label: 'AI credit' }

        // What you meter: each model's tokens, at your price per token.
        sonnet_input: { price: { amount: 0.000006 }, overhead_cost: 0.000003 }
        sonnet_output: { price: { amount: 0.00003 }, overhead_cost: 0.000015 }
        haiku_input: { price: { amount: 0.000002 }, overhead_cost: 0.000001 }
        haiku_output: { price: { amount: 0.00001 }, overhead_cost: 0.000005 }
    }
    plans: {
        pro: {
            default: true
            entitlements: {
                // Nothing included: every token is paid for with AI credits.
                sonnet_input: { limit: { credit: 'sonnet_input', value: 0 } }
                sonnet_output: { limit: { credit: 'sonnet_output', value: 0 } }
                haiku_input: { limit: { credit: 'haiku_input', value: 0 } }
                haiku_output: { limit: { credit: 'haiku_output', value: 0 } }
            }
            topups: {
                // 1,000 AI credits ($10) every month.
                monthly_ai_credits: {
                    credit: 'ai_credit', value: 1_000, included: true
                    resets: true, reset_sch: 'monthly:1'
                }
            }
        }
    }
}`);

await limitr.ensureCustomer('ada');

async function balance() {
    const credits = await limitr.remainingCredit('ada', 'ai_credit');
    return credits!.toFixed(1);
}
console.log('AI credits:', await balance());

// After each model call, record the provider's numbers. AI credits pay for the tokens.
async function record(model: string, input: number, output: number) {
    await limitr.allow('ada', `${model}_input`, input);
    await limitr.allow('ada', `${model}_output`, output);
    console.log(`${model}, ${input} in, ${output} out | AI credits:`, await balance());
}
await record('sonnet', 2_000, 800);
await record('haiku', 5_000, 1_200);

// What one AI credit buys, per model.
const sonnet = await limitr.creditExchange('ai_credit', 'sonnet_output', 1);
const haiku = await limitr.creditExchange('ai_credit', 'haiku_output', 1);
console.log(`1 AI credit: ${Math.round(sonnet!)} sonnet or ${Math.round(haiku!)} haiku tokens out`);

// Before a big call, check there are enough credits for its output.
const room = await limitr.check('ada', 'sonnet_output', 400_000);
console.log('room for 400k sonnet output:', room);

It prints:

AI credits: 1000.0
sonnet, 2000 in, 800 out | AI credits: 996.4
haiku, 5000 in, 1200 out | AI credits: 994.2
1 AI credit: 333 sonnet or 1000 haiku tokens out
room for 400k sonnet output: false

• Sonnet cost 3.6 credits (2,000 input tokens at $6 per million, 800 output at $30, so $0.036 at 1 cent a credit), and Haiku 2.2.
• Metering stays discrete. Each model's tokens are still their own meter, with their own price, cost, and events, so margin per model and line items work as before. The AI credit is only what the customer sees and spends.
• Limits of 0 with a hard mode mean nothing is included: calls run on credits until they run out. Check before a call that could be large, as the last line does. Make the limits soft to bill usage past the credits instead.
• remainingCredit(id, 'ai_credit') is the balance. remaining(id, entitlement) counts it too, converted into that entitlement's credit.
• Selling credits is a topup in ai_credit with a price. See Prepaid credits.

No code is needed in the policy for any of this, so it works the same in JSON, YAML, and TOML.

Pairs

Each entry in exchange is a pair: one unit of the name is worth value units of currency. A conversion follows pairs until it reaches rune, multiplying as it goes, so chains can be as long as you need.

exchange: {
    euro: { value: 1.14, currency: 'usd' }            // 1 euro is 1.14 USD
    ai_credit: { value: 0.01, currency: 'usd' }       // 1 AI credit is 1 cent
    team_credit: { value: 10, currency: 'ai_credit' } // 1 team credit is 10 AI credits
}

Two pairs are built in:

Pair	Meaning
rune: {}	The base unit.
usd: { value: 1, currency: 'rune' }	One US dollar is one rune.

• A priced credit needs no pair. Its price is its value in runes, so ai_token with price: { amount: 0.00002 } converts into anything else on its own.
• A pair wins over a price when a credit has both (see below).
• Validation rejects a pair that can't reach runes. A cycle converts to null.

Convert from your app with creditExchange(in, out = 'rune', value = 1), or credit_exchange(in, out, value) in Stof. A conversion with no path returns null. With the pairs above and a priced image credit at $0.04:

Conversion	Result
10 euro to usd	11.4
1 team_credit to usd	0.1
1 image to ai_credit	4
100 ai_credit to image	25

Tiered credits convert from a position in their tiers. The engine passes the customer's position when it prices a call; a plain creditExchange starts at the first tier.

Pricing credits through the exchange

A pair on a credit sets its value everywhere: what a call is worth, what's billed as overage, and what grants pay for it. So instead of a price per credit, you can price your models in AI credits, and keep the dollar value of an AI credit in one place:

policy: {
    exchange: {
        ai_credit: { value: 0.01, currency: 'usd' }           // 1 AI credit is 1 cent

        // Model prices, in AI credits per token
        sonnet_input: { value: 0.0006, currency: 'ai_credit' }  // 6 credits per 10,000 tokens
        sonnet_output: { value: 0.003, currency: 'ai_credit' }  // 3 credits per 1,000 tokens
        haiku_input: { value: 0.0002, currency: 'ai_credit' }
        haiku_output: { value: 0.001, currency: 'ai_credit' }
    }
    credits: {
        ai_credit: { label: 'AI credit' }

        // What each costs you stays on the credit
        sonnet_input: { overhead_cost: 0.000003 }
        sonnet_output: { overhead_cost: 0.000015 }
        haiku_input: { overhead_cost: 0.000001 }
        haiku_output: { overhead_cost: 0.000005 }
    }
}

Change ai_credit to 0.012 and every model's price rises 20%, with no other edits. Add a step to the chain (Ex. a premium credit between some models and ai_credit) to reprice a group of models at once.

Currencies

A currency is a credit with a pair, usually into usd:

exchange: {
    euro: { value: 1.14, currency: 'usd' }
    gbp: { value: 1.27, currency: 'usd' }
    cfa_franc: { value: 0.0017, currency: 'usd' }
}

Use one anywhere a credit goes: a spend cap in euros (credit: 'euro'), a grant in pounds, or a conversion for display. Keep rune at one US dollar (the default): prices, costs, caps, and events are all in runes, and they're simplest to read as dollars.

Live rates

Rates in the policy are fixed until the policy changes. For rates that move during the day, update them from your app or look them up when they're needed.

On a schedule. Your app fetches rates and sets the pairs with setExchangePair(name, value, currency):

import { Limitr } from '@formata/limitr';

const limitr = await Limitr.new(`
policy: {
    credits: { euro: { label: 'Euro' } }
}`);

// From a scheduled job, with rates from your provider
const rates = { euro: 1.16 };
await limitr.setExchangePair('euro', rates.euro, 'usd');
const usd = await limitr.creditExchange('euro', 'usd', 100);
console.log('100 euro in usd:', usd.toFixed(2));

// A pair that can't reach runes is refused, and the last rate stays
console.log('refused:', !(await limitr.setExchangePair('euro', 1.2, 'unknown')));
console.log('euro:', await limitr.exchangePair('euro'));

It prints:

100 euro in usd: 116.00
refused: true
euro: { value: 1.16, currency: 'usd' }

The same call reprices a model priced through the exchange (Ex. setExchangePair('sonnet_output', 0.0035, 'ai_credit')), and removeExchangePair(name) takes a pair out, so a priced credit goes back to its own price. In Stof, the engine functions are set_exchange_pair and remove_exchange_pair, and a capability can call them from inside the policy, with an allowed host.

On demand. Every conversion asks the exchange's get_pair(name) for a pair, so the policy can replace it to look rates up from your app and keep them for an hour:

exchange: {
    // Currencies with live rates
    live: ['eur', 'gbp']

    fn get_pair(name: str!) -> ExchangePair {
        const pair = self.get(name);
        if (!self.live.contains(name)) return pair;
        if (pair != null && pair.fetched > Time::now() - 1hr) return pair;

        const usd = await App?::usd_rate(name);           // your app's rate source
        if (usd == null) return pair;                     // keep the last rate we had
        if (pair != null) self.remove(name, shallow = false);
        const fresh = new { value: usd, currency: 'usd', fetched: Time::now() } on self;
        self.insert(name, fresh);
        fresh
    }
}

limitr.doc.lib('App', 'usd_rate', (currency: string) => rates[currency] ?? null);

get_pair is asked about every name a conversion passes through, credits included, so only look up the names you list and return the existing pair for everything else. See Call your app from a policy.

Grant strategy

When a customer has several grants that could cover a call, exchange.grant_strategy picks the order they're spent in:

Value	Spends first
expires_first (default)	The grant that expires soonest.
cheapest_first	The grant worth the least per unit, in runes.
valuable_first	The grant worth the most per unit.

See Grants and topups.