Reconciliations and the dependency stack

Why are reconciliations necessary in capital markets?

Capital markets have no unified data model. A trade passes through many hands before it settles: an execution venue, an OMS, a matching platform, a prime broker, a custodian, a fund administrator, a clearing house; and each holds a partial, differently shaped record of the same event. Translation is required at every hop.

Standardisation does exist, but only in pockets. Electronic venues and matching platforms impose a schema on the participants who use them, enabling some automation within those boundaries. However, as we set out in our piece on trade affirmations, even within a standardised venue the residue of unmatched and disputed items requires investigation in near real time, and any trade handled outside the platform has no representation inside it.

The same fragmentation exists within the firm. Front office, risk, accounting and operations run systems built at different times for different purposes, each with its own security master, valuation basis and cut-off. None is wrong; they answer different questions. But the firm still needs one defensible view of its positions, exposure, cash and P&L.

Moreover, these reconciliations are not independent. They sit in a dependency stack running from static data through transactions, positions, cash and valuation up to NAV. An unmapped instrument produces a transaction break, which produces a position break, which surfaces days later as a NAV discrepancy the administrator has flagged. Recs run bottom-up; breaks are investigated top-down, from the symptom back through several layers to a cause in a system the investigator may not even have access to.

What do reconciliations look like today?

The process can be modelled as four steps.

1. Data ingestion. Pulling the datasets to be compared: internal extracts, custodian and prime broker files, administrator reports - in whatever formats and on whatever schedules they arrive.
2. Break detection. Identifying how to pair a record from one source with its corresponding item from the alternative source: allocations, partial fills, netting, compressions and lifecycle events all mean the relationship is rarely one-to-one. Subsequently, comparing paired records field by field against tolerances. Both of these steps may require some data transformation.
3. Break classification. Gathering evidence, establishing the root cause, and deciding what kind of break this is, including whether it is a break at all.
4. Break resolution. Determining and executing the action that closes a true discrepancy: an email to the counterparty, an amendment in the OMS, a journal, an instruction to the administrator.

The first two steps have been the main focus of a large number of software vendors. They are deterministic, high volume and reproducible, and the industry has spent two decades building for them. Most solutions in this space focus on better data connectors, improved ways of setting up new recs, as well as some general workflow management capabilities that help teams collaborate on steps 3 and 4.

However, break classification and resolution are almost entirely manual today. The classification workflow is the most time-consuming as it is riddled with false positives. Most items surfacing from detection are not errors: they are timing differences, valuation basis differences, a late allocation, a known convention on a particular account. But establishing which one it is means assembling evidence from systems that do not talk to each other and traversing the dependency stack downwards from where the break appeared, while relying on tacit knowledge of what this account, this PM or this counterparty normally does. Resolution is manual by construction, because the action usually lives outside the reconciliation system entirely.

Sitting across all four steps is a fifth cost that rarely appears in anyone’s model: keeping the configuration alive. New accounts, new products, new file formats, tolerances that drift, mappings that break after a system migration. It is continuous, it is invisible, and it consumes a meaningful share of the team’s time.

The reason this has resisted automation is the one we described in our piece on trade affirmations. A rulebook can only encode the breaks somebody has already seen and written down, and new ones arrive every morning. The alternative - everyone agreeing a common data model - requires the whole market to move at once. The economics follow directly. Firms spend an order of magnitude more on the people investigating and resolving breaks than on the software that surfaces them. The licence is bought once; the team is paid every year, and it grows with the business rather than with the software.

What is our approach?

Our mantra is to build systems that finish the job end-to-end, and take human input only where it is truly needed. Four principles follow from that.

• Start where the biggest amount of effort is spent. Our initial focus was on break classification and resolution, not simply detection.
• Be deterministic where determinism is appropriate. Detection has to be reproducible and auditable. What changes is how the logic is configured, not how it is executed.
• Close the loop. A break classified as a false positive should change detection, not just get closed. Fewer items reaching an operator is a better outcome than resolving them faster.
• Retain human in the loop, but remove 95%+ of the effort. Operators should be able to see all of the relevant information, analysis and recommendations in an easy to digest, but comprehensive manner, so that they can take the final call in a matter of seconds.

Applied to the four steps we outlined:

• Data ingestion. Connect to systems and formats as they exist, without a canonical model as a precondition. The translation problem is the work, not something to be solved before the work starts.
• Break detection. Agents propose pairing logic, transformations and tolerances by observing the data and learning from how operators have resolved previous items. The engine then executes that configuration deterministically, so every run is reproducible and every break traceable to the rule that produced it. Judgment sits in the configuration; the execution stays mechanical.
• Break classification. This is where the agent does most of its work. It traverses the dependency stack from where the break surfaced down to where it originated, searching trade databases, affirmation platforms, mailboxes, static data and prior resolutions of similar items. It returns a proposed cause with the evidence attached, and every tool call it made is visible, so an operator can check the reasoning rather than take it on trust. Where the item is a known false positive, the finding is fed back into detection so the same item stops appearing.
• Break resolution. The proposed action is prepared and one click from execution, whether that is an email to a counterparty, an amendment payload for the OMS or an instruction to the administrator.

The fifth layer is handled the same way. Keeping the configuration alive stops being a separate maintenance project and becomes a by-product of the work: new accounts, new formats and drifting tolerances surface as breaks, get classified, and the resulting change flows back into the rec itself.

The compounding effect matters more than any single step. Conventions that have only ever existed in operators’ heads get written down as a consequence of resolving breaks, and the detection logic improves as the classification work accumulates. With Arch, the queue reaching the human, as well as the time spent per item, shrinks over time.

Reach out to find out more about how Arch handles reconciliations.