Govern Claude API calls (Messages API)
Govern calls made with the Anthropic SDK (@anthropic-ai/sdk in TypeScript, anthropic in Python;
client.messages.create) with a Limitr policy. Because the app makes each
call and runs its own tool loop, the policy can size each call exactly: offer only the tools the customer can use,
cap max_tokens at the tokens left under a hard limit, hold room for the whole call while it runs (so concurrent
calls can't overspend together), record the real usage when it returns, and answer refused tool uses with the reason.
Copy the recipe into your app and change TOOLS and TOKENS to your policy's entitlements. The tested copies, with
their tests, are in the repo: TypeScript
(with a live check) and Python. For agents built on the
Claude Agent SDK, see Claude Agent SDK.
The policy
One entitlement per governed tool and one that meters the model's tokens. Ex.:
policy: {
credits: {
call: { label: 'Tool call' }
ai_token: { label: 'AI token' }
}
plans: {
starter: {
default: true
entitlements: {
weather: { limit: { credit: 'call', mode: 'hard', value: 100, resets: true } }
ai_tokens: { limit: { credit: 'ai_token', mode: 'hard', value: 1_000_000, resets: true } }
}
}
pro: {
entitlements: {
weather: { limit: { credit: 'call', mode: 'hard', value: 5000, resets: true } }
refunds: {}
ai_tokens: { limit: { credit: 'ai_token', mode: 'soft', value: 20_000_000, resets: true } }
}
}
}
}
A hard ai_tokens caps each call's max_tokens at what's left; a soft one never caps it and bills past the
allowance. The event data carries model, input, cache_write, cache_read, and output for pricing (a credit per
model and direction, or a cost function; see AI token costs).
The recipe
- TypeScript
- Python
/**
* Limitr + the Claude Messages API (@anthropic-ai/sdk): size, decide, and meter every model call with your policy.
*
* Copy this file into your app and change TOOLS and TOKENS to your policy's entitlements. You keep making the
* calls and running the tool loop yourself; these helpers sit around each call and each tool use.
*/
import type Anthropic from '@anthropic-ai/sdk';
import type { Limitr, LimitrDecision } from '@formata/limitr';
/** The entitlement that decides each tool. Tools not listed here are always offered. */
export const TOOLS: Record<string, string> = {
get_weather: 'weather',
issue_refund: 'refunds',
};
/** The entitlement that meters the model's tokens (Ex. a hard monthly allowance, or soft and billed). */
export const TOKENS = 'ai_tokens';
/** Why a call was refused, in words the model (and your user) can act on. */
export function why(d: LimitrDecision): string {
switch (d.reason) {
case 'limit': return d.resets
? `${d.entitlement} limit reached, resets ${new Date(d.resets).toISOString()}`
: `${d.entitlement} limit reached`;
case 'governor': return `${d.entitlement} is rate limited, try again shortly`;
case 'cap': return `spend cap reached (${d.cap})`;
case 'held': return `${d.entitlement} is in use by other calls, try again shortly`;
default: return `${d.entitlement} not available (${d.reason})`;
}
}
export type Governed = { message: Anthropic.Message } | { refused: string };
/**
* One governed model call. The tools the customer can't use are left out, the call's input is counted, a hard
* token limit caps `max_tokens` at what's left, room for the whole call is held while it runs (so concurrent
* calls can't overspend together), and the real usage is recorded when it returns. Refused with the reason when
* the customer can't make the call; an API error releases the hold and rethrows.
*/
export async function governedCreate(
policy: Limitr,
client: Anthropic,
customer: string,
params: Anthropic.MessageCreateParamsNonStreaming,
): Promise<Governed> {
// Offer only the tools the customer can use right now.
const features = (await policy.features(customer)) ?? [];
const tools = params.tools?.filter((t) => {
const entitlement = 'name' in t ? TOOLS[t.name] : undefined; // toolsets have no single name
return !entitlement || features.includes(entitlement);
});
const request = { ...params, tools };
// Count the input (a free API call), then see what's left.
const { input_tokens: input } = await client.messages.countTokens({
model: request.model,
messages: request.messages,
system: request.system,
tools: request.tools,
} as Anthropic.MessageCountTokensParams);
const d = await policy.check(customer, TOKENS, input + 1, { model: request.model, input }, undefined, true);
if (!d.allowed) return { refused: why(d) };
// A hard limit caps the answer at what's left, less what other calls hold.
let maxTokens = request.max_tokens;
if (d.mode === 'hard' && d.allowance != null) {
maxTokens = Math.min(maxTokens, Math.floor(d.allowance - d.held - input));
if (maxTokens < 1) return { refused: why({ ...d, reason: 'limit' }) };
}
// Hold the most the call can use, run it, then record what it really used.
const context = { model: request.model, input };
const hold = await policy.reserve(customer, TOKENS, { value: input + maxTokens, context });
if (!hold) return { refused: why(await policy.check(customer, TOKENS, input + maxTokens, context, undefined, true)) };
try {
const message = await client.messages.create({ ...request, max_tokens: maxTokens });
const u = message.usage;
const event = {
model: message.model,
input: u.input_tokens,
cache_write: u.cache_creation_input_tokens ?? 0,
cache_read: u.cache_read_input_tokens ?? 0,
output: u.output_tokens,
};
await policy.settle(customer, TOKENS, hold, event.input + event.cache_write + event.cache_read + event.output, event);
return { message };
} catch (err) {
await policy.release(customer, TOKENS, hold);
throw err;
}
}
/**
* Decide one tool use from the model's response before you run it. Null when it may run (it's counted); otherwise
* the tool result to send back instead, with the reason, so the model can adapt.
*/
export async function decideToolUse(
policy: Limitr,
customer: string,
use: Anthropic.ToolUseBlock,
): Promise<Anthropic.ToolResultBlockParam | null> {
const entitlement = TOOLS[use.name];
if (!entitlement) return null;
const d = await policy.allow(customer, entitlement, 1, { tool: use.name }, undefined, false, true);
if (d.allowed) return null;
return { type: 'tool_result', tool_use_id: use.id, is_error: true, content: why(d) };
}
"""
Limitr + the Claude Messages API (anthropic): size, decide, and meter every model call with your policy.
Copy this file into your app and change TOOLS and TOKENS to your policy's entitlements. You keep making the
calls and running the tool loop yourself; these helpers sit around each call and each tool use.
"""
import math
from datetime import datetime, timezone
from typing import Any, Dict, Optional
import anthropic
from limitr import Decision, Limitr
# The entitlement that decides each tool. Tools not listed here are always offered.
TOOLS: Dict[str, str] = {
'get_weather': 'weather',
'issue_refund': 'refunds',
}
# The entitlement that meters the model's tokens (Ex. a hard monthly allowance, or soft and billed).
TOKENS = 'ai_tokens'
def why(d: Decision) -> str:
"""Why a call was refused, in words the model (and your user) can act on."""
entitlement, reason = d['entitlement'], d['reason']
if reason == 'limit':
if d.get('resets'):
resets = datetime.fromtimestamp(d['resets'] / 1000, timezone.utc).isoformat(timespec='seconds')
return f'{entitlement} limit reached, resets {resets}'
return f'{entitlement} limit reached'
if reason == 'governor':
return f'{entitlement} is rate limited, try again shortly'
if reason == 'cap':
return f"spend cap reached ({d.get('cap')})"
if reason == 'held':
return f'{entitlement} is in use by other calls, try again shortly'
return f'{entitlement} not available ({reason})'
def governed_create(policy: Limitr, client: anthropic.Anthropic, customer: str, **params: Any) -> Dict[str, Any]:
"""
One governed model call: {'message': message}, or {'refused': reason}. The tools the customer can't use are left
out, the call's input is counted, a hard token limit caps `max_tokens` at what's left, room for the whole call
is held while it runs (so concurrent calls can't overspend together), and the real usage is recorded when it
returns. An API error releases the hold and raises.
"""
# Offer only the tools the customer can use right now.
features = policy.features(customer) or []
request = dict(params)
if request.get('tools') is not None:
request['tools'] = [t for t in request['tools']
if TOOLS.get(t.get('name', '')) is None or TOOLS[t['name']] in features]
# Count the input (a free API call), then see what's left.
counted = client.messages.count_tokens(**{k: request[k] for k in ('model', 'messages', 'system', 'tools')
if request.get(k) is not None})
input = counted.input_tokens
context = {'model': request['model'], 'input': input}
d = policy.check(customer, TOKENS, input + 1, context, detail=True)
if not d['allowed']:
return {'refused': why(d)}
# A hard limit caps the answer at what's left, less what other calls hold.
max_tokens = request['max_tokens']
if d.get('mode') == 'hard' and d.get('allowance') is not None:
max_tokens = min(max_tokens, math.floor(d['allowance'] - d['held'] - input))
if max_tokens < 1:
return {'refused': why({**d, 'reason': 'limit'})}
# Hold the most the call can use, run it, then record what it really used.
hold = policy.reserve(customer, TOKENS, input + max_tokens, context)
if not hold:
return {'refused': why(policy.check(customer, TOKENS, input + max_tokens, context, detail=True))}
try:
message = client.messages.create(**{**request, 'max_tokens': max_tokens})
except Exception:
policy.release(customer, TOKENS, hold)
raise
u = message.usage
event = {
'model': message.model,
'input': u.input_tokens,
'cache_write': u.cache_creation_input_tokens or 0,
'cache_read': u.cache_read_input_tokens or 0,
'output': u.output_tokens,
}
policy.settle(customer, TOKENS, hold, event['input'] + event['cache_write'] + event['cache_read'] + event['output'], event)
return {'message': message}
def decide_tool_use(policy: Limitr, customer: str, use: Any) -> Optional[Dict[str, Any]]:
"""
Decide one tool use from the model's response before you run it. None when it may run (it's counted); otherwise
the tool result to send back instead, with the reason, so the model can adapt.
"""
entitlement = TOOLS.get(use.name)
if not entitlement:
return None
d = policy.allow(customer, entitlement, 1, {'tool': use.name}, detail=True)
if d['allowed']:
return None
return {'type': 'tool_result', 'tool_use_id': use.id, 'is_error': True, 'content': why(d)}
Running it: a tool loop
- TypeScript
- Python
import Anthropic from '@anthropic-ai/sdk';
import { governedCreate, decideToolUse } from './limitr-messages.js';
const client = new Anthropic();
async function chat(customer: string, messages: Anthropic.MessageParam[], tools: Anthropic.Tool[]) {
while (true) {
const res = await governedCreate(policy, client, customer, { model: 'claude-sonnet-5-5', max_tokens: 4096, messages, tools });
if ('refused' in res) return { refused: res.refused };
messages.push({ role: 'assistant', content: res.message.content });
const uses = res.message.content.filter((b) => b.type === 'tool_use');
if (uses.length === 0) return { message: res.message };
const results: Anthropic.ToolResultBlockParam[] = [];
for (const use of uses) {
results.push((await decideToolUse(policy, customer, use)) ?? await runTool(use)); // your tool code
}
messages.push({ role: 'user', content: results });
}
}
import anthropic
from limitr_messages import decide_tool_use, governed_create
client = anthropic.Anthropic()
def chat(customer: str, messages: list, tools: list):
while True:
res = governed_create(policy, client, customer, model='claude-sonnet-5-5', max_tokens=4096,
messages=messages, tools=tools)
if 'refused' in res:
return res
message = res['message']
messages.append({'role': 'assistant', 'content': message.content})
uses = [block for block in message.content if block.type == 'tool_use']
if not uses:
return res
# run_tool: your tool code
results = [decide_tool_use(policy, customer, use) or run_tool(use) for use in uses]
messages.append({'role': 'user', 'content': results})
What happens where
| Step | Limitr call | Why |
|---|---|---|
| Offer tools | features(customer) | The model never sees a tool the customer can't use (toolsets without a single name are always offered). |
| Count the input | client.messages.countTokens(...) | A free API call, so the call is sized by its real input. |
| Decide the call | check(customer, TOKENS, input + 1, …, true) | Refused with the reason before any model call. |
| Size the answer | max_tokens = min(asked, allowance - held - input) | Hard limits only. Room held by calls in flight counts. |
| Hold, run, record | reserve(input + max_tokens) → settle(real usage) | Concurrent calls can't overspend together; an API error releases. |
| Each tool use | allow(customer, TOOLS[name], 1, …, true) | Counted when it may run; otherwise a tool_result with is_error and the reason. |
Variations
- Skip the token count to save a round trip: estimate the input (Ex. characters / 4), or let the engine predict
it (
estimate_basison the entitlement; see Predict output tokens). The hold is then an estimate, andsettlestill records the real usage. - A minimum answer size: refuse when
max_tokenswould be under what's useful (Ex. 256) instead of 1. - Streaming: call
client.messages.stream(...)in place ofcreate, and settle fromfinalMessage()'s usage. Release the hold if the stream errors. - Downgrade instead of refuse: when what's left is low, switch
modelto a cheaper one before the call. - Tools priced by outcome:
reservebefore running the tool andsettlewith the real amount after. A hold spends a governor's tokens likeallow, so rate limits still hold. - Policy capabilities as tools:
policy.claudeTools(customer)returns the policy's capabilities as Claude tools, andpolicy.claudeToolUse(use, customer)runs one and returns itstool_result(see AI tools).
Notes
max_tokensis capped from a snapshot: two calls sized at the same moment both see the same allowance, but the holds make the laterreservefail if they don't both fit, and it's refused with the reason.- Output is capped by
max_tokens, so a call made throughgovernedCreatestays within a hard limit (unless the call's billed input differs from the token count).