Bulk Transaction Processing

Overview

The bulk transaction processing system enables importing multiple financial transactions through a single operation. It is built on top of the universal import system and integrates with the workflow engine for validation, entity resolution, and GL posting.

When to Use Bulk Processing

The import system is significantly faster for four or more records because it pre-caches reference data (vendors, customers, accounts, dimensions) and performs batch validation. Individual posts process each record from scratch with no caching.

---

Supported Transaction Types

All transaction types in the system are supported for bulk import. A single import can contain records of mixed types.

---

Entity Resolution

The importer resolves parties and references using flexible identifiers. Use whichever identifier you have available -- the system will find the matching record.

Vendor Resolution (for AP types)

Customer Resolution (for AR types)

Employee Resolution (for expenses and payroll)

Line-Level Resolution

All entity resolution uses pre-cached lookups, so referencing entities by name has minimal performance overhead in bulk imports.

---

Import Options

Duplicate detection uses the combination of transaction type and reference number. Updating existing posted transactions via import is not supported -- posted transactions are immutable financial records.

Defaults let you set values once that apply to every record. For example, setting a default currency or default dimensions avoids repeating the same values on every record.

---

Preflight Check

Before importing, run a preflight check to discover the required fields for a specific transaction type. Required fields are company-specific (configured in the workflow definition), so always check before building your records.

The preflight check returns:

• Required header fields
• Required line fields
• Optional header fields
• Optional line fields

Include a sample record to validate it against the full pipeline and get specific error messages before submitting the entire batch.

---

Synchronous vs. Asynchronous Processing

Imports are automatically routed based on record count:

Why Asynchronous?

Each transaction involves extensive validation, entity resolution, and GL posting. For large batches, synchronous processing would exceed timeout limits. Asynchronous mode processes records in batches and reports progress as it goes.

Monitoring Progress

For asynchronous imports, poll the import status to track progress:

• In progress: Shows percentage complete, processed records, imported count, and error count
• Completed: Shows final counts for imported, skipped, and errored records, plus duration

Concurrency Limit

Only one large import (over 50 records) per organization runs at a time. Small imports (50 or fewer records) are unaffected and can run in parallel.

---

Error Handling

Per-Record Isolation

Each record is processed independently. One failed record does not block the others. Successfully imported records are not rolled back when other records fail.

Common Errors

Recovery from Partial Failures

When some records fail in a batch:

1. Check the error details for specific issues per record
2. Fix the problematic records
3. Re-submit with "skip duplicates" to skip already-imported records
4. Successfully imported records from the first run remain committed

---

Batch-Level Approval (Optional)

Organizations can optionally enable batch-level approval for bulk imports:

1. User submits import
2. Records are stored and a single approval workflow is created for the entire batch
3. Approver reviews a summary of the batch
4. On approval, all records are processed with batch optimizations
5. On rejection, nothing is imported

Approval Configuration

When approval is required, the import status progresses through: pending approval, importing, completed.

---

Common Workflows

CSV File Import

1. Run a preflight check to discover required fields
2. Parse the CSV and map columns to transaction fields
3. Determine legal entity, transaction type, and default values
4. Build records from each CSV row
5. Preview a summary (record count, total amount, date range) before submitting
6. Submit the import
7. Review results (imported, skipped, errors)

Repetitive Transaction Patterns

When you describe a batch of similar transactions (such as "12 monthly rent invoices"), the system can generate records programmatically, show you a preview, and submit via the import workflow.

Data Migration from External Systems

For migrations from systems like QuickBooks or Xero:

1. Import master data first -- Vendors, customers, employees, and accounts with external IDs
2. Import transactions using external IDs -- Reference entities by their external identifiers
3. Use "skip duplicates" -- Allows safe re-runs if the migration is interrupted

---

Pre-Submit Checklist

Before submitting a bulk import, verify:

1. Legal entity ID is correct for all records
2. Required dimensions are included
3. Vendors and customers exist for all referenced parties
4. Account numbers are valid for the entity
5. Tax codes are appropriate
6. Dates are in YYYY-MM-DD format
7. Reference numbers are unique to avoid duplicates
8. Currency is set (defaults to USD if not specified)
9. Payment method and terms are included where required
10. Preview has been reviewed with totals before submitting

---

Key Capabilities

• Mixed transaction types in a single import operation
• Flexible entity resolution by ID, name, external ID, or tax ID
• Pre-cached lookups for fast batch processing
• Preflight validation to discover required fields before importing
• Automatic sync/async routing based on batch size
• Per-record error isolation so one bad record does not block the rest
• Duplicate detection with skip or error behavior
• Optional batch-level approval with configurable risk lanes
• Safe re-runs for interrupted migrations
• Default values to avoid repeating common fields across records