Skip to main content

Test a policy

A pricing change is a code change: it can lock customers out or give features away. Test it the same way. The engine is the same in tests as in production, so tests run against the real thing in milliseconds.

Tests in TypeScript​

This uses Node's built-in test runner (npx tsx --test pricing.test.ts). Vitest and Jest work the same way.

import { Limitr } from '@formata/limitr';
import { test } from 'node:test';
import assert from 'node:assert/strict';

// In your repo: the policy file you ship, Ex. readFileSync('./policy.stof', 'utf8').
const policy = `
policy: {
    credits: {
        request: { label: 'API request' }
        ai_token: { label: 'AI token', price: { amount: 0.00002 } }
    }
    plans: {
        free: {
            default: true
            entitlements: {
                api: {
                    limit: {
                        credit: 'request', value: 1000
                        resets: true, reset_inc: 1day
                    }
                }
            }
        }
        pro: {
            entitlements: {
                api: {
                    limit: {
                        credit: 'request', value: 100_000
                        resets: true, reset_sch: 'monthly:1'
                    }
                }
                chat: {
                    limit: {
                        credit: 'ai_token', mode: 'soft', value: 2_000_000
                        resets: true
                    }
                }
                sso: {}
            }
        }
    }
}`;

// A fresh engine per test, so tests can't affect each other.
const engine = () => Limitr.new(policy);

test('the policy is valid', async () => {
    const [ok, message] = await (await engine()).valid();
    assert.ok(ok, message);
});

test('free customers get 1,000 API requests a day', async () => {
    const limitr = await engine();
    await limitr.ensureCustomer('t1');
    assert.equal(await limitr.allow('t1', 'api', 1000), true);
    assert.equal(await limitr.allow('t1', 'api', 1), false);
});

test('free customers have no AI chat or SSO', async () => {
    const limitr = await engine();
    await limitr.ensureCustomer('t2');
    assert.equal(await limitr.check('t2', 'chat', 1), false);
    assert.equal(await limitr.check('t2', 'sso'), false);
});

test('pro bills chat past 2M tokens', async () => {
    const limitr = await engine();
    await limitr.ensureCustomer('t3', 'pro');
    const bills: number[] = [];
    limitr.addHandler('test', (name, value) => {
        if (name === 'meter-overage') bills.push(JSON.parse(value as string).overage_price);
    });
    assert.equal(await limitr.allow('t3', 'chat', 2_500_000), true);
    assert.deepEqual(bills, [10]);                   // 500k tokens at $20 per million
});

test('upgrading keeps the usage counted so far', async () => {
    const limitr = await engine();
    await limitr.ensureCustomer('t4');
    await limitr.allow('t4', 'api', 600);
    await limitr.setCustomerPlan('t4', 'pro', false);
    assert.equal(await limitr.value('t4', 'api'), 600);
    assert.equal(await limitr.remaining('t4', 'api'), 99_400);
});
  • A fresh engine per test keeps tests independent. Creating one takes a few milliseconds.
  • Test what the plan promises: what's allowed, what's denied, what's billed, and what happens on plan changes.
  • Test the events you bill from. Collect them with a handler and assert on the amounts.
  • Run them in CI next to valid(), so a pricing change can't ship broken.

Time-based behavior​

Resets don't run on a timer; the engine notices a new period on the next call. To test a reset, give the test policy a short period (reset_inc: 1s) and wait past it, or test the logic around it with resets(id, entitlement), which returns when the current period ends.

Tests in Stof​

Policies are Stof documents, so tests can live next to the policy, in Stof, and run with the Stof CLI. They need the engine's source, from the Limitr repo (here at ./limitr):

pricing.test.stof
import 'limitr/src/spec/mod' as self.LimitrTypes;

Limitr policy: {
    credits: {
        request: { label: 'API request' }
    }
    plans: {
        free: {
            default: true
            entitlements: {
                api: {
                    limit: {
                        credit: 'request', value: 1000
                        resets: true, reset_inc: 1day
                    }
                }
            }
        }
    }
}

#[test]
fn free_api_is_capped() {
    const p = self.policy;
    p.create_customer('t_free_api', 'free');
    assert(p.allow('t_free_api', 'api', 1000));
    assert_not(p.allow('t_free_api', 'api', 1));
    assert_eq(p.remaining('t_free_api', 'api'), 0);
}

#[test]
fn policy_is_valid() {
    assert(self.policy.valid());
}
stof test pricing.test.stof
stof test --leaks pricing.test.stof
  • Limitr policy: { ... } gives the policy the engine's type, so self.policy.allow(...) works directly.
  • Tests in a document run concurrently against the same document, so give each test its own customer IDs.
  • --leaks runs the tests one at a time and fails any that leave objects behind. Run it when a policy has functions (cost functions, rules, notifications): see Write policies in Stof.
Last updated on