This is part 3 of Building the Agent Platform. Part 1: Anatomy of a governed finance-agent platform. Part 2: Agent blocks.
There's a picture of AI agents that lives in demos: one brilliant general agent that "handles finance." After a year of running agents on real companies' books, I can report that production looks nothing like that. It looks like a team of narrow specialists — each with a job description, a budget, explicit permissions, and a manager to escalate to — plus an enormous amount of institutional knowledge encoded as thresholds, guards, and things-we-learned-the-hard-way.
This is a walk through that team: what the agents at Artifi actually do, with real confidence thresholds and real incidents. Parts 1 and 2 of this series covered the runtime and the composition machinery; this part is the payload. I've written it for both audiences at once — finance readers will recognize the work; engineers will recognize the patterns — because the whole point of finance agents is that those two groups now share a system.
The bill processor: reading is the easy part
The flagship agent processes supplier bills arriving by email or upload. The naive version of this is "OCR plus GPT" — extract vendor, amount, date, post it. The production version is a pipeline where extraction is maybe a fifth of the work, and the rest is deciding whether to believe the extraction.
First: is this bill even ours? Forwarded emails, group companies, personal purchases — bills arrive addressed to all sorts of entities. Before anything posts, a deterministic relevance check runs in priority order: our tax ID appearing in the document confirms it (confidence 0.95); a fuzzy match on our legal name (≥85% of tokens, legal suffixes stripped) confirms it; address components vote with weights (postal code 0.35, city 0.25, street 0.25, country 0.15, threshold 60%); a known employee as sender counts (work email 0.85, personal 0.75); an explicitly different addressee blocks processing. Nothing AI-fancy — a scoring function — but it's the difference between an agent and a liability, because posting someone else's bill into your VAT return is not a cute failure mode.
Second: who is this vendor? Vendor resolution runs through tiers: a run-local cache, then a shared memory — one "vendor brain" that the bill, bank, and card agents all read and write, keyed by a normalized counterparty fingerprint — then a validated sender-memory, then database search, and finally, only for genuine ambiguity, a small-model disambiguation that sees a numbered list of up to 150 existing vendors and must answer with a number or NONE (cost: about a tenth of a cent; instruction: prefer NONE when unsure).
That word "validated" carries a production scar. The agent used to trust its memory of "bills from this sender belong to vendor X" unconditionally. Then someone forwarded a batch of bills from different vendors, and every one of them got pinned to the vendor of the first bill ever learned from that sender. Six bills, one wrong vendor. The fix is general and worth engraving: memory keyed on ambient context (who sent it) must be cross-checked against document-extracted identity (whose bill it is) before use. Learned shortcuts are hypotheses, not facts.
Third: which expense account? This is where finance judgment lives, and where we learned that model choice is an accounting decision: the smaller model scattered the same SaaS vendor across different GL accounts bill by bill — cloud hosting on one account this month, another the next — which technically balances and practically ruins comparability. The fix was three-layered: a stronger model for classification; a vendor-default pin (set "this vendor → this account" once on the vendor record and every future line is deterministically pinned — no model consulted); and for unpinned vendors, a learning loop that recalls this vendor's past service-to-account decisions as prior context, writes back new ones after validation (so hallucinated accounts can never be learned), deduplicates, and caps the memory at 25 entries per vendor.
Fourth: the tax trap. VAT extraction has a failure mode that nearly always ends in a wrong ledger: misread a cost component as a tax amount, and the system helpfully finds a tax code to justify it. Our rule set is deterministic and humble: compute the effective rate from the extracted amounts, accept a tax code only within 0.3 percentage points of a real rate (23.96% matches the 24% code), and treat airline taxes, fuel surcharges, and booking fees as cost components, never VAT. The old tolerance was 1.0 points — wide enough that a misread airline invoice (32.45 over 345 ≈ 9.4%) force-matched a 9% code and grossed up every line. Tolerance tightening as a bug fix.
Fifth: refuse to balance by force. If extracted lines don't sum to the total, the agent adds a rounding adjustment only within 5 cents. Any larger gap routes to a human — no plug entries, ever. An agent that "makes it balance" is an agent that hides extraction errors inside your P&L.
All of these gates land in the same place: a review queue. Five conditions route a bill to a person — overall extraction confidence below 0.85 (scored additively: keywords, total found, vendor found, invoice number, date, line items), an entity mismatch, any credit memo (always reviewed), a line-total mismatch, ambiguous tax. The review screen shows the agent's extraction beside the source PDF; on approval, the bill posts through the same workflow gateway as everything else. The design principle: the first three triggers doubt the extraction; the last two doubt the accounting. A confidently misread invoice sails past confidence scoring — you need correctness gates, not just confidence gates.
The bank agent: the human's answer is the classification
Bank and card transactions flow through a second specialist, and its architecture inverts what most people expect from "AI classification."
The cheap, deterministic checks run first. A refund pre-detector looks for a recently posted opposite-direction transaction — same bank account, same merchant, amount within ±0.5%, original within 90 days — and books a proper refund against the original instead of inventing a new transaction. This exists because of a specific bug: a vendor refund once became a phantom customer payment, complete with a phantom customer record. The general lesson: LLMs classify what a thing looks like; only the ledger knows what a thing is in relation to other things. Relationship checks must precede classification.
Then memory (past classifications of this counterparty fingerprint), and only then a small model choosing among twelve transaction types — with a prompt that explains each type's accounting purpose rather than listing keywords, a redesign forced by "transfer between own accounts" getting classified as a vendor payment. On a first real statement run, 14 of 39 lines needed the model; on the second run, zero — memory had taken over. That's the cost curve you want: intelligence up front, muscle memory forever after.
Some lines legitimately stump every tier. Those get batched into a single review request, and here's the inversion: the lines are not posted with a guess awaiting correction — they are not posted at all until the human answers, and the answer becomes the posting. The accountant writes, in a form or free text, "the 3,200 to Maria is salary, the 5,000 is a dividend" — a strict parser (that degrades to "unclear" rather than guessing) turns that into classifications, salary settles the payroll accrual, the dividend posts as an owner distribution. Then the learning rule with a subtlety I love: patterns where all lines resolved identically get memorized; the person who receives both salary and dividends is deliberately never memorized and will be asked about every time. Ambiguity, once discovered, is respected.
Two hard guards ride on top, both born from incidents: card payments to government or tax authorities are always reviewed regardless of what any model says (they settle liabilities, not expenses, and models flip-flop on them); and a credit from a known vendor that no refund-matcher could pair goes to review instead of becoming revenue. Guards are cheaper than corrections.
Reconciliation: six passes and one refusal
Matching payments to invoices is the agent that most rewards engineering discipline, because its failure mode — the wrong match — is quiet. The matcher runs six passes in strict order of trustworthiness: exact amount-and-window matches (confidence 1.0), reference-number matches (0.95), invoice numbers found inside payment descriptions and vice versa (0.92–0.95), FX-tolerant approximate matches (0.85–0.95, scaled by how far the amounts diverge, hard-capped at 5%), combinatorial grouping for one-payment-many-invoices (0.50–0.95, with limits to keep the combinatorics sane), and finally — for the dregs — a small-model fuzzy pass that lands at 0.70, which is below the auto-apply line by design.
That line is 0.85: above it, matches apply automatically; below it, they queue for a person. And one refusal encodes more accounting wisdom than any prompt I've written: when matched amounts differ in the same currency, the agent never writes off the difference automatically — not even a cent-level one. A same-currency variance means a wrong match, an undisclosed discount, or a short payment, and auto-writing it off would mask exactly the situations a person needs to see. Cross-currency differences, by contrast, post automatically as FX gain/loss — that's arithmetic, not judgment. Knowing which differences are arithmetic and which are information: that's the job.
The month-end close: an agent that is forbidden to fix anything
The most institutionally interesting agent runs the close — as a saga of 13 phases with dependency ordering (bank completeness, card completeness, expense reports, subledgers, payroll, reconciliation, GL validation, tax integrity… through to the close package), where each phase runs a registry of checks with fixed severities: blocking checks gate the close; advisory checks warn and proceed. Clients can configure whether a check applies to them; they can never weaken a blocking check into an advisory one. Governance you can configure away isn't governance.
Three design rules make it trustworthy:
The verifier never fixes. When a blocking check fails, the close agent doesn't repair the data — it opens a request to the responsible specialist agent, waits, and re-checks (up to three iterations, then it pauses for a human rather than failing). Separation of duties, applied to software: the entity that certifies is never the entity that remediates.
Dry-run is the default at every layer. Config, launcher, scheduler, UI — everything defaults to a mode where all checks run, all findings are recorded, and nothing is delegated or closed. Going live is an explicit, per-organization flip. The first production dry-run — on a real company — surfaced five false-failure bugs in the checks themselves. That's the point: the dry-run's first job is to close-test the closer.
The LLM is a junior analyst with no signing authority. A handful of checks use a model to review manual journal entries and scan for anomalies — and those checks are permanently advisory, so a model outage can never block a close. Better: their findings pass through a fabrication guard that drops any finding containing a number that doesn't appear in the supplied data. Models turned out to judge well and invent arithmetic while explaining — so we keep the judgment and strip the invented numbers. And when everything is green, the agent still doesn't close the period: it submits the close through the approval lane, and a human clicks. The reopen action, incidentally, requires the highest approval tier — reopening a closed period is the more dangerous direction, and for a while, embarrassingly, it wasn't gated at all.
The money-movers: where determinism is non-negotiable
Payments are where "the agent got creative" stops being an anecdote and starts being a regulator's question, so the payment agents contain no model calls at all.
The payment proposal agent assembles batches deterministically: due AP invoices and approved expense reimbursements, grouped by (payment method, currency, source bank) because a SEPA file and an ACH file cannot mix; one line per invoice for reviewability; capped at 200 lines. Which of the company's banks pays a given vendor isn't guessed — it was learned by the bank agent from the first observed real payment and stored on the vendor record (currency-matched), a nice example of one agent's observation becoming another agent's configuration. The batch then goes to approval: any manager for normal batches, senior approval above $50,000 with a 48-hour escalation, line-level exclusion in the approval screen (uncheck two suspicious lines, approve the rest), and a hard rule that the submitter cannot approve — separation of duties survives the automation. And the GL posts only when the bank statement confirms execution — the statement, not the transmission, is the source of truth.
The collection agent initiates charges (cards, direct debit) only for contracts explicitly configured for active collection — the default for any unconfigured contract is do nothing, because the safe default for touching customers' money is inaction. SEPA direct debit revenue posts only when the settlement webhook confirms it, days later — not when the charge is initiated. Dunning retries at 3, 7, and 14 days, each retry a scheduled event tied to its transaction and attempt number, capped at three attempts.
The two chaser agents — one asks vendors' side of the house for missing bills (payments with no invoice), one asks employees for missing receipts — share an elegant non-design: escalation with no state machine. The age of the item is the state: 0–7 days gets a friendly tone, 7–21 firm, 21+ urgent (configurable, with per-bracket recipient overrides — 45+ days can go to the CFO). Per-item cooldowns prevent nagging; the email is drafted by a small model with a hardcoded fallback template. My favorite bug in the whole system lives here: with 91 outstanding payments, the model-drafted email hit its token limit and truncated the list mid-sentence — so above 20 items, the agent switches to generating a CSV attachment. LLMs summarize; files enumerate. Know which one you need.
Around them, the routine machine: the billing agent generates invoices from contracts (each contract in its own database transaction so one failure can't take down the run; idempotent via the advancing next-billing-date; "already billed" explicitly reported as skipped, not failed, to keep failure dashboards meaningful), and each created invoice fans out — by event, not by call — to the delivery agent (PDF, email, logged) and the collection agent in parallel. A customer missing a tax profile doesn't crash billing: the contract is skipped, a request goes to the master-data agent, and the callback re-bills automatically. The delivery agent's contribution to the pattern catalog: a customer with no email is a skip, not a failure — a data gap and an error are different things, and conflating them buries real errors under noise.
The patterns, extracted
Across eleven agents, the same ideas keep recurring. If you're building agents for finance — or hiring someone who will — this is the checklist I'd interview against:
- Deterministic first, model second, human third. Every agent runs its cheap, reliable checks before its clever ones, and escalates what neither can settle. The model is a middle layer, not the architecture.
- Confidence gates AND correctness gates. Confidence catches what the model doubts; invariants (lines must sum, rates must exist, relationships must reconcile) catch what the model wrongly believes.
- The human's answer is input, not absolution. Reviews drive posting — and consistent answers become memory while ambiguous ones deliberately don't.
- Learn with validation, cap what you learn, respect discovered ambiguity. Every learning loop has a write-back guard and a size limit; every memory is a hypothesis until cross-checked.
- Refuse the convenient fiction. No plug entries to force balance, no auto-write-offs in same currency, no posting salary lines without a real accrual to settle, no revenue on charge initiation. Each refusal keeps a human-shaped hole where judgment belongs.
- Separate the doer from the certifier from the approver. The close agent can't fix; the payment proposer can't approve; the reopen needs more authority than the close.
- Events chain the team. Billing → delivery + collection; bank posting → reconciliation → bill chasing. No orchestrator god-object; each agent does its job and announces the result, and the announcement is the next agent's trigger.
The demo fantasy is one agent that understands finance. The production reality is a dozen narrow agents that each understand one job plus the accumulated scar tissue of every way that job goes wrong — with humans wired in exactly where the scars taught us to put them. It's less romantic. It closes the books.
Andrew Rudchuk is the founder of Artifi. All thresholds, incidents, and mechanisms in this article are from the platform's production documentation. Part 1 covers the runtime architecture; part 2 covers the block and pipeline composition system.