This is part 2 of Building the Agent Platform. Part 1: Anatomy of a governed finance-agent platform. Part 3: What production finance agents do all day.
Part 1 of this series described the runtime of a finance-agent platform: events, execution modes, the write gateway, humans as first-class objects. This part is about the problem that showed up after all of that worked.
Our agents were monoliths. The bills-collector agent — the one that chases vendors for missing invoices — was one Python class holding every decision: which payments count as orphaned, how urgency escalates with age, who receives the email, what the email says, when a reminder is too soon. It worked. And then design conversations with prospective customers kept producing the same sentence in different accents: "That's almost exactly what we need, except…"
Except we escalate to the CFO after 45 days, not 21. Except payments to our payroll provider should never be chased. Except we want our own risk model to score each payment before anything sends. Every "except" had exactly two answers: change the product for everyone, or fork the code for one customer. Both answers are wrong. Every client receives every change the moment it deploys, and any per-org customization beyond a config file requires a developer fork — that's the sentence from our own internal design doc that started this work.
The answer we built is agent blocks: agents decomposed into small, typed, individually-tested functions, wired into pipelines by data, customizable per organization by patching — never forking. This article is the full design, including the three sandboxed block types that let customers inject their own logic into our agents without putting their code on our infrastructure, and the sharp edges we found.
A block is a function with a contract
A block is a small async function registered into a catalog. Its registration declares four things that make composition safe:
- a typed parameter model (Pydantic) — validated before anything runs;
reads— the named state fields it consumes;writes— the named state fields it produces;effectful— whether it touches the outside world (sends email, calls an API, writes business tables).
The signature never varies:
async def my_block(ctx: BlockContext, params: MyParams, state: dict) -> dict:
And the contract is enforced with deliberate strictness: the returned dict must contain exactly the keys declared in writes — an undeclared key is an error, a missing declared key is an error, both fail the stage. That sounds pedantic until you've debugged a pipeline where some stage quietly smuggled an extra field into shared state and a later stage started depending on it. Contract drift is how composable systems rot; we made it a runtime exception instead.
A pipeline is data, and it's validated like a program
A pipeline is a YAML file next to the agent's config — a linear list of stages, each naming a block, its params, and error policy:
stages:
- id: query_payments
block: bills.query_orphaned_payments
params: { lookback_days: "$config.lookback_days|90" }
- id: classify_and_group
block: bills.classify_by_age
- id: send_emails
block: bills.send_reminder_emails
on_error: halt
retry: 1
timeout_seconds: 300
Stages communicate through a shared, named-field run state — a typed blackboard. And because every block declares its reads and writes, the pipeline is validated at apply time like a program: every field a stage reads must be written by an earlier stage or seeded by the runtime (trigger, config, item). A wiring mistake is a rejected config, not a 2 a.m. incident.
Each stage gets a timeout (default 300s), a retry count, and an error policy — halt or skip. Every stage execution writes an audit row: block name and version, duration, which fields went in and out (shapes, not payloads — {"rows": 47}), a result preview, the error if any. When a customer asks "why did last night's run email only three vendors?", the answer is a table scan, not log archaeology.
Two more design points that carry weight:
Loud failure, never fallback. If a pipeline is missing or invalid, the run fails visibly. It does not quietly fall back to the legacy code path or — worse — to an LLM improvising the workflow. A broken pipeline must be a page, not a silent degradation. We wrote that rule down early and it has paid for itself every time something drifted.
Iteration is explicit. A stage can declare over: <list-field> to run once per item, with results accumulated into a named collection. Per-item writes are visible only within the iteration — only the accumulated collection crosses into later stages. Without that rule, loops become state-pollution machines.
Effects go through a guard, and the guard is the database
The dangerous stages are the effectful ones — the block that actually sends 47 dunning emails. Two mechanisms make them safe to retry and safe to run concurrently:
The effect guard. Before every external action, an effectful block must claim an idempotency key: an INSERT ... ON CONFLICT DO NOTHING RETURNING into a dedicated ledger table keyed by (organization, agent, key). Row inserted → proceed. Key already there → skip, and record that you skipped. This is database-level idempotency, not application state — two concurrent instances of the same agent on the same org cannot double-send, no locks required.
Content-derived keys. The key for a reminder email is derived from the recipient and the payment IDs it covers — not from a timestamp — so it's stable across retries and parallel instances. Calendar-scoped effects bucket the key by UTC date, so "at most once per day" allows a legitimate re-send tomorrow. And one deliberately asymmetric choice: when an effect times out (did the email send? unknown), the claim is not released — the system biases against double-sending money-adjacent communications, accepting that a failed-but-claimed effect waits for the next day's bucket.
Migration by canary, not big bang
We didn't rewrite our agents into pipelines. We picked one — the bills collector — decomposed it into seven stages (resolve entities → query payments → classify by age → resolve recipients → compose → send → stamp reminders), and ran the pipeline version against the legacy class with a parity test: same inputs, assert same outcomes. The helper logic (age-bracket classification, cooldown checks) was moved to the blocks and the legacy agent now delegates to those same functions — one copy of the logic, guaranteed agreement, no drift between old and new paths.
Rollout is a per-organization flag: execution_mode = 'pipeline' on the agent's definition row. One org flips, everyone else keeps the battle-tested class; rollback is an UPDATE statement; deploy-time sync explicitly never overwrites a database opt-in back to the file default. Boring migrations are the only migrations that work on financial infrastructure.
Patches, not forks: per-organization customization
Here is the payoff. An organization's customization is a patch — a small list of typed operations keyed on stable stage IDs, stored per (org, agent). The canonical pipeline ships with the product; the patch belongs to the org; the effective pipeline is computed by applying one to the other. Four operations:
{"op": "set_params", "stage": "query_payments", "params": {"lookback_days": 120}}
{"op": "disable", "stage": "csv_attachment"}
{"op": "insert", "after": "classify_and_group", "stage": { ...new stage... }}
{"op": "replace", "stage": "compose_emails", "replacement": { ... }}
The guardrails matter more than the ops:
disableis permission-gated by the author. Only stages the canonical author markeddisableablecan be removed. A customer cannot disable the stage that computes a field every later stage depends on — the platform author decides which vertebrae are load-bearing.- Two-layer validation. At save time, the patch is applied to the current canonical and the result fully re-validated — a bad patch is rejected before it's stored. At load time (every dispatch), the stored patch is re-applied against the canonical as deployed — because the product evolves, and a patch written against last month's pipeline might not fit this month's. If it no longer applies, the run fails loudly; it never silently falls back to the unpatched canonical, because "your customization stopped applying and we ran the default over your objection" is the worst possible behavior for a finance workflow.
- Preview-first workflow. Patch updates take a
confirmedflag; the unconfirmed call returns the validated effective pipeline without saving. Look, then leap.
This is the upgrade-safety contract from Part 1's architecture, made concrete: the customer's logic lives in data (patches), the product's logic lives in code (blocks and canonical pipelines), and the boundary between them is validated on every single run.
Dry-run as a first-class mode
Every pipeline can execute with dry_run=true: all stages run, all state flows, all audit rows are written (flagged as dry-run) — and zero effects happen. No emails, no business-table writes, not even idempotency-ledger rows.
The implementation detail worth copying is the belt-and-suspenders: it would be easy for one effectful block to forget its dry-run branch. So the effect guard itself — the choke point every effect must pass — raises a dedicated DryRunViolation exception when called in dry-run mode. A forgotten dry path becomes a loud stage failure instead of a real email to a real vendor. Effectful blocks return realistically-shaped stubs ({"sent": false, "dry_run": true}) so downstream stages exercise their real logic.
One honest caveat we documented rather than hid: LLM-judgment blocks still call the model during dry-run — a preview of judgment has to actually run the judgment, and it costs real tokens. Dry-run guarantees zero side effects, not zero cost.
If this pattern sounds familiar from Part 3's period-close saga — dry-run by default, effects only after explicit opt-in — that's not a coincidence. In finance software, "show me what you would do" is not a feature request; it's the adoption path. Nobody flips a system to live that couldn't first prove itself in shadow mode.
Three sandboxed ways to inject customer logic
Patches that reorder and reparameterize our blocks cover a lot. The remaining "except…" cases need customer logic. We shipped three special block types, deliberately ordered from cheapest and most predictable to most powerful:
1. Expression blocks (CEL). For filters, mappings, and guards, patches can embed expressions in Google's Common Expression Language: filter this payment list to amounts over 500 excluding vendor 42, flag anything older than the config threshold. Three properties make CEL the right tool: it's non-Turing-complete — no loops, no I/O, no imports, so an org-authored expression can be evaluated safely on shared infrastructure without a sandbox escape surface; it's compiled at patch-save time — a syntax error is rejected at authoring, an invalid expression can never reach a production run; and it's fast and deterministic. One domain-specific rule we enforce in review: money comparisons use >=/<, never == — the values cross a float boundary and exact equality on floats is a bug generator.
2. LLM blocks. One scoped judgment step inside an otherwise deterministic pipeline: params are a prompt, an input field, an output field, and a required JSON schema for the result. No tools, no conversation, no agency — the model reads one field and produces one typed answer ("rank these overdue payments by likely collection difficulty"). The output is validated against the schema; one retry on violation, then the stage fails. This inverts the usual agent architecture — instead of an LLM orchestrating tools, the pipeline orchestrates and the LLM is a function call — and in a finance context that inversion is exactly right. Tier selection (small/large model) and token caps are params; costs are tracked per-purpose like every other model call in the platform.
3. HTTP callout blocks. The escape hatch for real custom code: the pipeline POSTs stage data to a customer-registered HTTPS endpoint — their Lambda, their Cloud Run, their infrastructure — and the response merges back into pipeline state. Their proprietary risk model participates in our workflow without their code ever running on our machines. The security envelope took as long to design as the feature:
- Endpoints are registered with a server-minted secret, encrypted at rest, returned exactly once and never retrievable again (re-registering rotates it). Requests carry a timestamp and an HMAC-SHA256 signature over timestamp + body, so the receiving endpoint can verify both origin and freshness.
- Field allow-listing is enforced twice: the fields an endpoint may receive are declared at registration, and every execution re-checks the requested payload is a subset — so a later patch can't quietly leak new fields to an endpoint whose owner never consented to them.
- SSRF protection: HTTPS only, every resolved IP must be public (re-resolved per call, no DNS caching), redirects not followed, 30-second timeout, 1 MiB response cap.
- Callouts are idempotent per content-hash per day, and — same asymmetry as email — a timed-out callout does not release its claim: double-send is worse than delayed-send.
The preference order is stated in the docs and enforced in review: expression before LLM before callout. Determinism before judgment before delegation. Every step down that ladder costs predictability, and predictability is the product.
What this actually buys
Step back and the shape is simple. The platform ships verbs — tested, versioned, contract-checked blocks. Organizations write sentences — pipelines and patches, in data, validated like programs, previewable and dry-runnable. Custom logic enters through three doors, each with an explicit safety envelope, and never through "let me just fork the agent."
For the lonely finance-AI architect — the single engineer hired to automate a finance function — this is the difference between owning a bespoke codebase that dies with their tenure and configuring a substrate that upgrades under them. For us as a vendor, it's the answer to "every client wants a variation" that doesn't multiply codebases. And for an auditor, every stage of every run — including the customer's own patches, expressions, and callouts — leaves the same audit rows as our first-party logic.
What's deliberately not built yet: fully org-authored pipelines distributed from customer git repos (designed, not shipped), and LLM blocks with bounded tool access. Both wait for the same reason everything here waits — each new degree of freedom needs its safety envelope designed first. That sequencing — capability only after containment — is, I've come to think, the actual discipline of building agent platforms for serious domains.
Part 3 leaves the machinery and walks through the agents themselves: what eleven production finance agents actually do all day, with their real confidence thresholds, review triggers, and the incidents that shaped them.
Andrew Rudchuk is the founder of Artifi. The block engine, patches, dry-run mode, and all three special block types described here are in production; the design docs they're drawn from are the platform's internal plans, lightly edited for publication.