This is part 1 of Building the Agent Platform, a three-part long read on how a production finance-agent platform actually works. Part 2: Agent blocks. Part 3: What production finance agents do all day.
Everyone is building AI agents this year. Very few of those agents are allowed to touch a general ledger — and the distance between "an agent that drafts things in a sandbox" and "an agent whose output an auditor will examine" is almost entirely architecture, not prompts.
For the past year I've been building Artifi, a finance platform where autonomous agents process supplier bills, classify bank transactions, reconcile payments, run billing, and prepare the month-end close — in production, on real companies' books. This article is the architecture write-up I wish I'd had when I started: the runtime, the control surfaces, the cost mechanics, and the failure modes, with real numbers throughout. Nothing here is theoretical; every mechanism exists because something broke without it.
The one constraint that shapes everything
A finance agent's write target is a double-entry ledger. That single fact cascades into every design decision, because a ledger has properties most agent playgrounds don't:
- Mistakes compound. A misposted invoice doesn't just sit there; it flows into VAT returns, aging reports, and the P&L.
- Everything must be attributable. "Who posted this, based on what document, approved by whom" is not a nice-to-have; it's what an audit is.
- Some actions are irreversible in practice. You can reverse a journal entry, but you cannot un-send a payment or un-file a tax return.
So the platform's core principle is: agents never write directly. Every write goes through the same gateway humans use, gets risk-scored, and either flows (low risk), waits for one approval (moderate), or waits for multi-party approval (high). The agent doesn't know it's being governed; it just calls submit() like everyone else. That's the whole trick, and the rest of this article is the machinery that makes it work at acceptable cost and reliability.
Events, not cron
The runtime is a queue. Five kinds of sources feed events into one table: inbound email, schedules, connector syncs (bank data arriving), downstream rules, and agents themselves (more on that later). A single long-lived worker — the event processor — consumes the queue and dispatches agents.
Why a queue instead of cron jobs calling agent scripts? Because a queue gives you, for free: retry with backoff (ours: 60s → 300s → 900s, three attempts), an audit trail of every trigger, deduplication (SHA-256 of the payload, unique per organization and event type — and deliberately only active events block duplicates, so a failed event can be retried), backpressure, and the ability to investigate and replay any failure. Cron gives you none of that; it gives you a log line.
The latency trick worth stealing: Postgres LISTEN/NOTIFY. An AFTER INSERT trigger on the events table fires a notification; the worker holds a dedicated listen connection and wakes immediately, with a 30-second fallback poll in case the listen connection drops. Event pickup went from ~2.5 seconds (5-second polling) to ~100ms, with no new infrastructure — no Redis, no Kafka, just the database you already have. For a finance workload's volumes, this is enough, and every component you don't add is a component that can't fail.
Every event and every agent run is a state machine with honest states. An event can be pending, processing, completed, failed, ignored (no agent wanted it), expired (7 days), or duplicate. An agent instance can be pending, running, completed, failed, cancelled, timeout — and one more that carries most of the governance weight: waiting_approval. When an agent finishes its work but its submitted writes are sitting in an approval lane, the instance isn't done — it's waiting for a human. The workflow engine completes or fails the instance when the human decides. That bidirectional linkage (agent runs know their workflows, workflows know their agent) is what lets you answer "what did this agent actually change, and who signed it?" in one query.
Agents are files, not database rows
Every agent is a directory in the repo: a config.yaml and a prompt.md. The config declares what the agent is allowed to be:
agent_type: bank_transaction_processor
model_tier: standard # fast / standard / advanced
execution_mode: code # llm / code / hybrid
max_concurrent_runs: 3
timeout_minutes: 30
allowed_workflows:
- transaction.post
- bank_statement_line.match
allowed_tools:
- search
- list_entities
- submit
trigger_events:
- connector.data_received
- bank_statement.manual_import
On startup the worker syncs files to the database. Two fields are never overwritten by sync — the agent's API key and its enabled flag — so operators keep runtime control while everything else lives in git. This buys you the things engineers already trust: version control for prompts, code review for behavior changes ("this PR makes the bill processor stop auto-creating vendors" is a reviewable diff), and identical definitions in every environment.
Notice what the config amounts to: least privilege for a non-human worker. allowed_workflows is the agent's write permission set. allowed_tools is its read surface. trigger_events is when it's allowed to exist. max_concurrent_runs is how many of it may exist at once. If you've done IAM design, this is familiar — and that familiarity is the point. Agents are colleagues with narrow job descriptions, not a general intelligence with a database connection.
Three execution modes, because $3 per bank statement is absurd
The most consequential lesson in the whole platform, and it's about money: most finance work is deterministic, and paying an LLM to do deterministic work is engineering malpractice.
Agents run in one of three modes:
Early on, the bank transaction processor was an LLM agent: ~$3 per statement run. Rewritten as a code agent that calls a small model (Haiku) only for the genuinely fuzzy 10% — "which vendor is 'AMZN Mktp DE*2K4' really?" — it costs about $0.01 per run. That's a 99.7% cost reduction with higher reliability, because the deterministic 90% now behaves identically every time. Two other agents (configuration, master data) got the same treatment for ~90% savings. The execution_mode field means switching an agent between modes is a config change, not a rewrite.
The heuristic that emerged: LLM for reading, code for arithmetic, human for judgment above a threshold. Document extraction, classification of ambiguous text — LLM. Matching, posting rules, date math, batch assembly — code. Anything the agent isn't confident about, or that crosses a risk line — route to a person (the next section). Systems that get this split wrong in either direction fail differently: all-LLM systems are expensive and non-deterministic where they least can afford it; all-code systems can't read an invoice.
Model tiers follow the same logic: fast (Haiku-class) for extraction and classification, standard (Sonnet-class) for judgment-heavy work like GL account selection, advanced reserved for the rare genuinely hard reasoning. Each targeted small-model call costs around $0.001.
The 664-tool problem
Here's a scaling problem nobody warns you about. Our MCP server exposes roughly 664 tools — the full finance surface: master data, transactions, banking, reconciliation, reporting. Each tool definition costs 500–1,000 tokens of context. Give an agent all of them and you've spent 300K–650K input tokens per request before the agent has read a single word of the actual task — plus you've bought worse tool selection and slower inference.
The fix is boring and essential: allow-lists. The bill processor needs exactly 9 tools; with the allow-list applied, its tool context is ~4.5K tokens. That's a two-orders-of-magnitude reduction, applied at the SDK level by computing the complement (everything not allowed becomes explicitly disallowed) and additionally blocking 16 built-in tools — Bash, file access, web fetch — that no autonomous finance agent has any business holding.
Two production lessons from this machinery, offered as warnings:
The naming trap. The SDK namespaces tools as mcp__<server_name>__<tool>. The server name comes from one string in the server code; the config key comes from another. When one said Arfiti and the other arfiti, every prefix mismatch failed silently — meaning all tool filtering quietly turned off and every agent saw every tool at 10–100x the token cost. No error, just a bigger bill. If you build tool filtering, build an assertion that it's actually filtering.
Turn count matters as much as token count. Early bill processing made ~21 individual tool calls per invoice — each one a full conversational turn with its own overhead. The fix was a bundled context tool: one call returning all the AP reference data an invoice needs (vendors, accounts, tax codes, payment terms, posting profiles). Combined with allow-lists, bill processing went from $2+ to roughly $0.20–0.30 per invoice. When you control both sides of the tool interface, design tools around the workflow's data needs, not around your database tables.
Humans are an object type, not an interruption
The piece of this architecture I'd defend most fiercely: human-in-the-loop is not a UI afterthought — it's a first-class record in the system, called an agent request.
When an agent hits something it can't or shouldn't decide — a bill it doesn't trust, a bank line it can't classify, a vendor missing required data — it doesn't fail, guess, or park the work in a log. It creates a request: typed (approval, verification, master data, configuration...), routed (to a person, with assignment fallbacks from explicit assignee to org owner to any admin), tracked (due in 14 days, expires in 90, reminders counted), and resumable — the request records which agent instance created it, so when it completes, a callback event routes to the creator, which picks the work back up.
The subtle part — and the redesign that made the whole thing click — is that human responses drive action. Completing a review isn't acknowledgment; it's input. When the bank processor batches 12 unclassifiable statement lines into one review request and the accountant answers — in a structured form or in plain text — the agent parses that answer (free text goes through a strict small-model parse that degrades to "unclear" rather than guessing) and posts the lines accordingly. Salary lines settle the payroll accrual. Dividend lines post as owner distributions. And then the agent learns: description patterns whose lines were all resolved the same way get memorized for next month; ambiguous patterns — the same person receiving both salary and dividends — are deliberately not memorized and will be asked again. The human isn't approving the agent's work; the human is the highest-quality classifier in the system, invoked at the exact moments the cheaper classifiers are unsure.
The same object type covers agent-to-agent delegation: the bill processor finding an unknown vendor delegates creation to the master-data agent; billing delegates missing tax profiles; anything unclassifiable goes to a triage agent that only routes (never executes recovery) and carries a hop counter capped at 2, after which a human gets it. One table, one lifecycle, one inbox — workflow approvals and agent requests appear as one task list to the user.
Everything fails; design for the specific failures
A year of production distilled into the failure catalog and its countermeasures:
Deploys strand work. A mid-run deploy once left delegation events stuck in processing for 15 hours — vendor creations that silently never happened. Now startup runs orphan recovery: instances still marked running are moved to timeout (visible, slot freed), and all stuck processing events are reset to pending with their retry counters intact. Complementing it, a lost-delegation detector: a callback re-run that finds no active request and none completed in the last 30 minutes is allowed to re-escalate — exactly once.
Agents loop. Early this year, a hallucinated entity ID (the LLM invented an entity the organization didn't have) produced four full delegation loops before manual intervention. Three circuit breakers came out of that incident: a global limit (50 completed callback-triggered runs per org per 30 minutes — counting only callback events, because new-work delegations are legitimate fan-out, and only completed runs, because failures are already bounded by retries); a per-run flag so a callback re-run gets exactly one chance and never re-delegates; and a one-hour dedup window on identical configuration requests. Plus the root-cause fix: the write gateway now validates entity IDs exist. Layered defenses, because any single layer has false negatives.
LLMs claim success. An LLM agent can end its run saying "done!" having done nothing. Failure detection is layered: pattern-matching on output ("tool not found", "connection failed"), verifying that tool calls actually carried the MCP prefix, checking the SDK's connection flag, and the blunt final check — a "successful" run with zero tool calls is overridden to failed. Beyond that, the auto-complete safety net verifies claims: when an agent forgets to close its delegation request, the processor checks whether the vendor it was asked to create actually exists by name — if yes, complete the request with the real entity ID; if no, mark it honestly failed so recovery re-escalates. And the master-data agent's prompt is explicitly forbidden from claiming "I scheduled a retry" — LLM agents have no retry mechanism, and letting them believe otherwise produces convincing fiction.
Bursts overwhelm. Concurrency limits exist at three levels (a global LLM semaphore, per-org caps so one busy org can't starve others, per-agent run limits — the reconciliation agent runs strictly single-instance to prevent double-matching). The important refinement: when the per-org gate trips, the event is deferred and re-queued, not failed. Early versions dropped burst overflow terminally; "defer, don't drop" is the difference between a throttle and a data-loss bug.
Made-up contact data escapes. When an agent must create a customer from a bank line with no email, it uses a reserved placeholder domain that never resolves, flags the record's metadata with exactly which fields are placeholders, opens a human review request — and every outbound-communication path checks those flags before sending, with the email layer refusing the placeholder domain unconditionally as the final belt. Made-up data is sometimes unavoidable; escaping made-up data never is.
What the operator sees
Every agent run is an inspectable record: what triggered it, a progress log, structured output, token usage, model used, estimated cost to four decimals, and links to every workflow it submitted. Queue statistics, failed-event lists, retry buttons, request queues, per-org monthly cost rollups. The debugging loop for "why didn't my bill get posted?" is four queries: instance → its trigger event → its open requests → retry. None of this is glamorous, and all of it is the difference between an agent platform and a demo. If I could give one sizing heuristic: we've spent meaningfully more engineering on the around — queueing, recovery, observability, permissions — than on the agents themselves. That ratio is correct, and if yours is inverted, you have a demo.
The numbers, in one place
- Event pickup latency ~100ms (LISTEN/NOTIFY) vs ~2.5s (polling); 30s fallback poll
- Retry backoff 60/300/900s, 3 attempts; events expire after 7 days
- ~664 registered tools; 9 allowed for the bill processor; ~4.5K tokens filtered vs 300–650K unfiltered
- Bill processing ~$0.20–0.30/invoice (was $2+); bank statement run ~$0.01 (was ~$3, −99.7%); targeted small-model call ~$0.001
- Loop breaker: 50 completed callback runs / 30 min / org; triage hops capped at 2
- Human requests: due +14 days, expire +90; lost-delegation recovery window 30 min
The takeaways
If you're building agents for any domain where mistakes have ledgers — finance, healthcare, legal:
- One write gateway, no exceptions. Governance you bolt on per-agent will be inconsistent per-agent.
waiting_approvalis a state, not a UI. Model human gates in the runtime's state machine or they'll be invisible to it.- Execution mode is an economic decision. Classify each workflow: read (LLM), arithmetic (code), judgment (human). The bill for getting this wrong arrives monthly.
- Context is the cost center. Tool allow-lists and bundled context tools cut our per-task cost ~10x before any model got cheaper.
- Humans are the best classifier you have. Build the request object, parse their answers, learn from consistent ones, and re-ask about ambiguous ones.
- Design for the failure you'll actually have: deploys mid-run, loops via self-triggering, LLMs claiming success, bursts, and fabricated data. Each needs its own countermeasure, and most need two.
Part 2 covers what happened when single-purpose agents weren't customizable enough: agent blocks — typed, composable pipeline stages with per-organization patches, dry-run mode, and sandboxed extension points. Part 3 walks through the production agents themselves, thresholds and all.
Andrew Rudchuk is the founder of Artifi, where finance teams run their function through Claude on a governed ledger. The architecture described here runs in production; the docs it's drawn from are the platform's actual internal guides.