Skip to content

Case Study: Automating Cross-System Data Reconciliation

From 15 Hours of Manual Data Entry to a Generic Workflow Architecture

Overview

A small five-person company described a recurring operational problem: approximately 15 hours per week were spent manually entering data, reconciling spreadsheets, and copying the same information between a CRM, an invoicing system, and Google Sheets.

The problem was not that any individual system was necessarily inadequate.

The problem was the space between the systems.

People were acting as the integration layer—checking records, copying values, identifying discrepancies, deciding what needed attention, and manually keeping multiple systems aligned.

This case study explores whether Execura could address that underlying problem without attempting to replace the existing CRM, invoicing platform, or spreadsheet.


The Original Problem

The reported workflow looked approximately like this:CRM │ ├── Customer / project information │ ▼ Human │ ├── Copy information ├── Check values ├── Compare records ├── Update spreadsheet ├── Check invoice └── Resolve discrepancies │ ├──────────────► Invoicing System │ └──────────────► Google Sheets

The company estimated that this manual coordination consumed around 15 hours every week.

The important observation was that the work was not primarily about creating new business data.

It was about ensuring that existing data agreed across systems.


The Hypothesis

Instead of replacing the existing applications, we formulated a different hypothesis:

Could Execura become the workflow and reconciliation layer between existing systems?

The conceptual architecture became:External System A │ ▼ Execura │ ├── Retrieve data ├── Map response ├── Normalize workflow data ├── Compare objects ├── Detect discrepancies │ ├── MATCH ───────────────► Continue │ └── MISMATCH │ ▼ Human Exception │ ▼ Human Resolution │ ▼ Continue Workflow │ ▼ Audit + Reporting

This preserved the existing systems while introducing a controlled orchestration layer.


What We Built

The investigation was deliberately divided into independent capabilities rather than immediately building a CRM-specific or invoicing-specific integration.

M1 — External API Response Mapping

The first requirement was to retrieve information from external systems and make the relevant response data available to subsequent workflow steps.

The implementation reused Execura’s existing API/webhook infrastructure rather than creating a dedicated reconciliation integration.

The canonical workflow path was verified to:

  1. Execute an external API request.
  2. Receive the response.
  3. Apply configured response mappings.
  4. Store the resulting values in workflow state.
  5. Persist those values correctly.
  6. Make them available to subsequent workflow steps.

A persistence defect was discovered during hostile verification and corrected so the aggregate mapped response could survive workflow reconstruction.


M2 — Generic Object Reconciliation

The next question was whether Execura could compare objects without knowing anything about invoices, CRM records, or a particular industry.

A generic reconciliation service was introduced.

For example:{ "left": { "customer_id": "C-100", "amount": 5000, "quantity": 10 }, "right": { "customer_id": "C-100", "amount": 5000, "quantity": 8 } }

The workflow can produce a deterministic result such as:{ "status": "MISMATCH", "matched": [ "customer_id", "amount" ], "differences": [ { "field": "quantity", "expected": 10, "actual": 8 } ] }

The reconciliation engine supports explicit field mappings and nested paths while distinguishing important cases such as:

  • missing values
  • null
  • undefined
  • false
  • 0
  • empty strings
  • different data types

The implementation deliberately avoided:

  • fuzzy matching
  • tolerance-based matching
  • arbitrary expressions
  • eval
  • domain-specific invoice logic
  • CRM-specific assumptions

The objective was a generic reconciliation primitive.


M3 — Human Exception Handling

A mismatch is not necessarily an error that software can resolve automatically.

Sometimes a human must decide what happens next.

Therefore, the workflow was extended so that:MATCH ↓ Continue automatically

while:MISMATCH ↓ Create exception ↓ Assign human ↓ Workflow waits ↓ Human resolves ↓ Workflow continues

The exception lifecycle was integrated with Execura’s existing workflow, assignment, notification, audit, and persistence infrastructure.

The implementation was then subjected to hostile testing.

This uncovered issues around assignment creation failures and duplicate assignments.

Those were corrected using both application-level protection and database-level uniqueness constraints.

The resulting system protects against duplicate active assignments, including concurrent creation attempts.


M4 — Reporting and Observability

Once reconciliation existed, the next question was:

Can the organization actually measure what is happening?

Rather than creating a separate reconciliation database, reporting was derived from durable workflow execution and assignment records.

The reporting model provides metrics including:

  • total reconciliation executions
  • MATCH count
  • MISMATCH count
  • exception count
  • resolved exceptions
  • unresolved exceptions
  • resolution duration
  • reconciliation rate
  • exception rate
  • execution history
  • trends over time

The system also maintains the relationship between workflow execution, reconciliation result, human exception, and resolution.

This makes the process observable rather than leaving reconciliation as an invisible automation step.


M5 — Reliability and Recovery

At this point the capability appeared functional.

That was not sufficient.

The next milestone deliberately attempted to break it.

The complete chain was tested against:

  • external API failures
  • malformed responses
  • missing fields
  • null mappings
  • persistence failures
  • duplicate delivery
  • concurrent assignment creation
  • concurrent human resolution
  • workflow restart
  • WAITING-state reconstruction
  • partial persistence
  • repeated resolution
  • cancellation
  • retry scenarios

The hostile review discovered seven correctness defects.

Examples included:

Duplicate human resolution

A previously resolved assignment could incorrectly appear successful when resolved again.

Incorrect workflow advancement

The workflow engine could automatically advance after resolution without sufficiently verifying the current durable workflow state.

Non-transactional variable persistence

Process-variable updates could partially persist during a failure.

Swallowed database errors

An exception handler was suppressing database errors that should have been surfaced.

Incorrect audit identifier

Exception audit logging used a step identifier where a UUID-backed step-instance identifier was required.

Null mapping failure

A null response mapping could cause an unexpected execution failure.

All seven defects were corrected and verified.

The final M5 suite contained 31 dedicated reliability tests, with the existing reconciliation suite also passing.


M6 — Real-World Validation

The final validation stage asked a different question:

Can a real operator actually use the capability?

Three generic scenarios were evaluated:

Scenario 1 — Matching Records

Two systems provide equivalent customer/project data with different field names.

Example:System A System B customerId ───────► client_id amount ───────► total currency ───────► currency

Expected result:MATCH

The workflow continues automatically.


Scenario 2 — Single Discrepancy

Two systems provide the same identifier but different amounts.System A: amount = 5,000 System B: amount = 4,800

Expected result:MISMATCH ↓ Human Exception ↓ Human Decision ↓ Workflow Continues


Scenario 3 — Multiple Discrepancies

The reconciliation engine was tested with multiple simultaneous differences, including:

  • identifier differences
  • amount differences
  • missing values
  • null values

The result preserves the individual differences rather than reducing the outcome to a simple boolean.


The Critical Finding

The M6 validation produced an important distinction.

The backend reconciliation capability is implemented and tested.

However, the operator-facing experience is not yet complete.

The investigation found:

Capability

Result

Reconciliation engine

Implemented

Reconciliation API

Implemented

Reliability/recovery

Verified

Automated tests

Passing

Workflow-builder reconciliation step

Not yet exposed

Exception resolution UI

Not yet exposed

Reconciliation reporting UI

Not yet exposed

Workflow export/import

Partial

The existing backend therefore cannot honestly be presented as a finished low-code operator feature.

At the current stage, a user would need to interact with raw APIs or manually inject workflow configuration to exercise the capability.

That is an important product distinction.


Verification Results

The reconciliation backend was subjected to extensive automated verification.

Reconciliation test suite

114 / 114 tests passed

Additional regression testing

56 / 56 tests passed

Combined verified tests

170 / 170 tests passed

The verification covered:

  • object reconciliation
  • nested mappings
  • response mapping
  • exception creation
  • assignment lifecycle
  • multi-assignee behavior
  • duplicate prevention
  • concurrency
  • cancellation
  • execution identity
  • reporting
  • restart/recovery
  • persistence failures
  • authorization
  • tenant isolation
  • audit consistency

What This Demonstrated

The experiment established that the original problem can be represented as a generic workflow problem rather than a CRM or invoicing problem.

The reusable architecture is:Connect ↓ Map ↓ Normalize ↓ Reconcile ↓ MATCH / MISMATCH ↓ Automate or Escalate ↓ Human Resolution ↓ Resume ↓ Audit ↓ Measure

This architecture can potentially apply to many situations where information is distributed across systems and humans currently act as the reconciliation layer.

Examples include:

  • customer/project data synchronization
  • operational record reconciliation
  • inventory comparisons
  • payment verification
  • document metadata verification
  • compliance checks
  • data-quality workflows
  • inter-system record validation

The implementation deliberately avoids embedding these domains into the core engine.


What We Did Not Claim

The investigation did not claim that Execura eliminated the reported 15 hours per week.

There was no controlled baseline measurement performed with the original company.

Therefore, claims such as:

  • “15 hours saved”
  • “X% productivity improvement”
  • “Y% ROI”

would be unsupported.

The evidence establishes that Execura can technically model and execute the underlying reconciliation workflow.

It does not yet establish the real-world time savings for that particular company.


Current Product Status

Backend capability

Validated and tested.

The generic reconciliation engine can:

  • retrieve external data
  • map responses
  • compare structured objects
  • identify differences
  • route mismatches to humans
  • pause and resume workflows
  • preserve evidence
  • recover from failures
  • prevent duplicate operations
  • provide reporting data
  • maintain audit history

Operator experience

Not yet complete.

The next development stage is therefore not another reconciliation algorithm.

It is the creation of the minimum operator-facing vertical slice required to expose the already-proven backend capability through Execura’s low-code workflow experience.


Key Lesson

The most important result of this case study was not the reconciliation algorithm.

It was the discovery of the boundary between a working backend capability and a usable product capability.

Building the engine was only half the problem.

A real product must allow an operator to configure the workflow, understand the result, act on exceptions, and verify what happened without requiring direct database access or developer intervention.

That distinction prevents a common failure mode in software development:

Mistaking implemented functionality for a finished product.

This case study therefore remains open at the product-experience layer while the underlying reconciliation architecture has been independently tested and validated.

Leave a Reply

Discover more from Sowft | Transforming Ideas into Digital Success

Subscribe now to keep reading and get access to the full archive.

Continue reading