The Marriage of Compliance and Data, Part 2: Building Systems That Could Actually Keep Up

Part 1 of this series traced compliance’s early, clumsy attempts to capture data about its own programs, relying on hotline volumes and training completion rates that told you very little about whether misconduct was actually being deterred or caught. It also described the gap that emerged once regulators started expecting companies to actually use their data: the systems compliance functions had simply weren’t built to deliver a timely, integrated picture of program effectiveness. Part 2 picks up there, with the shift toward purpose-built data systems designed for expedited auditing and continuous monitoring, and the specific pressures that made building those systems unavoidable rather than optional.

Why “Periodic” Auditing Stopped Being Good Enough

For a long stretch, compliance auditing followed a periodic model: pick a sample of transactions, a business unit, or a geography, conduct a review once a year or once every few years, and issue findings well after the fact. This approach had an obvious structural weakness. By the time an annual audit surfaced a problem, the underlying misconduct could have been running unchecked for months, sometimes years, accumulating exposure the entire time. Sampling-based auditing also meant that most of a company’s actual transaction volume was never reviewed at all, leaving compliance functions to extrapolate from a small slice of data and hope it was representative.

Three converging pressures made this model untenable. First, the sheer volume and velocity of digital transactions kept growing, making periodic sampling look increasingly like reviewing a tiny fraction of a much larger, faster-moving river. Second, data privacy regulation, beginning in earnest with the GDPR and cascading into state and sectoral privacy laws elsewhere, meant companies suddenly had to demonstrate real-time visibility into how personal data moved through their systems, not just an annual snapshot. You couldn’t credibly tell a regulator you were compliant with data-handling obligations if your only evidence was a sampling exercise conducted once a year. Third, the cybersecurity threat environment made clear that a breach discovered during a periodic review, rather than at the moment it occurred, could mean months of unauthorized access before anyone even knew there was a problem to investigate. Each of these pressures pushed in the same direction: from periodic, backward-looking review toward continuous, forward-looking monitoring.

The Rise of Continuous Monitoring and Integrated Dashboards

In response, compliance functions began investing in systems designed to pull data continuously rather than episodically. Transaction monitoring systems, originally built for anti-money laundering compliance in financial institutions, spread into broader compliance use cases: flagging unusual payment patterns, screening counterparties against sanctions and watch lists in near real time rather than at onboarding only, and surfacing statistical outliers in expense reports or vendor payments that a human reviewer would never catch by manually sampling a subset of records.

Alongside these monitoring systems came the compliance dashboard, an attempt to finally solve the fragmentation problem described in Part 1. Rather than hotline data living in one platform, training records in another, and due diligence files in spreadsheets, compliance functions began building integrated views that pulled multiple data streams into a single place, letting a chief compliance officer actually see, in one view, where hotline volume, investigation resolution times, training completion, and third-party risk scores stood at a given moment, rather than reconstructing that picture manually every time a board update was due.

This mattered enormously for the audit function specifically. Continuous, system-driven monitoring meant auditors could shift from sampling a small percentage of transactions to reviewing effectively all of them, using automated rules to flag exceptions and anomalies for human review rather than relying on a human to manually spot them within a limited sample. An audit that once took months to plan and execute could increasingly be run as an ongoing, always-current process rather than a discrete, backward-looking project.

Key Risk Indicators Became the New Vocabulary

This period also saw compliance functions develop a more sophisticated vocabulary for what they were actually measuring, borrowing from operational risk management the concept of key risk indicators, forward-looking metrics meant to signal an emerging problem before it fully materialized, as distinct from lagging indicators that only confirmed a problem after the fact. A rising number of expense report exceptions flagged by an automated system, for instance, could function as a leading indicator worth investigating, rather than simply another data point logged and filed away.

This shift from purely descriptive, lagging metrics toward genuinely predictive, leading indicators represented real conceptual progress for the profession, even before AI entered the picture. It meant compliance data was finally being used the way regulators had started asking companies to use it: not just to document what had already happened, but to anticipate what might happen next and intervene before it did.

Where This Still Fell Short

Even with continuous monitoring and integrated dashboards, real limitations remained. Most of these systems still relied heavily on predefined rules: if a transaction exceeded a certain dollar threshold, or matched a specific pattern, it got flagged. That approach catches known risks effectively, but it struggles badly with novel risks that don’t match any rule someone thought to write in advance. It also generates significant false-positive volume, burying compliance analysts in alerts that need to be manually triaged, many of which turn out to be nothing. The systems had gotten dramatically faster and more integrated than the fragmented, periodic approach that preceded them, but they were still fundamentally reactive to patterns humans had already anticipated, not capable of recognizing genuinely new forms of risk on their own.

That limitation, the gap between fast, rules-based monitoring and something closer to genuine real-time risk intelligence, is exactly what the arrival of AI is now starting to close. That’s where Part 3 of this series picks up.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *