Beyond the Hype: Practical Lessons on Building With AI in Regulated Industries

AI promises the earth, but only disciplined teams turn it into dependable value in regulated spaces.

Mark Watson is CTO at ComplyAdvantage, a RegTech helping financial institutions detect and prevent financial crime with AI-driven tools.

In this conversation, Mark cuts through the noise to share what works, what doesn’t, and why measurement, explainability, and culture matter as much as models.

From reducing false positives to building auditable AI workflows, Mark explains how to move beyond hype towards trustworthy automation that stands up to regulation and makes life easier for customers.

The Shift From Machine Learning To Practical AI

Mark started working with machine learning long before the current wave of excitement, and he’s clear that something genuinely new arrived with modern language models.

Large models can process vast volumes of unstructured text, which is central to financial crime detection and adverse media screening.

That shift has enabled faster iteration, simpler retraining cycles, and better results on real-world data than older, hand-tuned approaches.

Recent UK market analysis also points to broad adoption across financial services, with regulators emphasising governance, validation, and accountability alongside innovation.

That context helps teams set expectations, secure buy-in, and align AI projects with emerging rules and supervisory focus.

Inside The Product: Where AI Works Today

ComplyAdvantage uses AI across the lifecycle: inside customer-facing products, within engineering, and for internal productivity.

In adverse media analysis, for example, older NLP models might take weeks to retrain; newer approaches can be updated in hours and at far lower operational cost.

That allows faster responses to new typologies without waiting for long modelling cycles.

It also unlocks “good enough to ship” improvements that compound when you verify and measure them relentlessly.

Automating What Should Be Automated

In compliance operations, the biggest wins often come from triaging noise rather than chasing novelty.

High-sensitivity systems generate false positives that must be reviewed, explained, and either released or escalated.

AI can now handle a meaningful slice of that workload with deterministic, auditable steps, cutting queue lengths and time to resolution.

The net effect is fewer unnecessary holds for legitimate customers and more capacity for analysts to focus on the edge cases that really matter.

Sector guidance recognises this direction of travel, highlighting where digital transformation can strengthen AML effectiveness when deployed responsibly.

Move Fast, But Only As Fast As The Tech Allows

Mark’s rule of thumb is simple: run at the speed the technology can safely support.

Don’t assume autonomous agents can self-improve beyond narrow, well-scoped tasks today.

Instead, stitch together clear, deterministic workflows, add human oversight where judgement or explainability is required, and measure everything.

That mindset mirrors supervisory updates that stress testing, validation, and explainability as non-negotiables for AI in financial services.

Design For Explainability From Day One

In regulated industries, decisions must be reconstructed and explained long after they’re made.

Teams that scatter logic across multiple tools often discover too late that the evidence is fragmented, inconsistent, or hard to reconcile.

Mark advocates for a single-platform approach where possible, with consistent data foundations, immutable audit trails, and a common reasoning layer.

That way, when a regulator asks “why,” you can show inputs, transformations, and outcomes in one coherent line of sight.

Guidance on explaining AI-assisted outcomes reinforces the need for accessible, human-centred explanations alongside technical logs.

"Don’t get over your skis.

Automate the deterministic steps, instrument everything, and keep humans in the loop where the obligation can’t be fully satisfied by software."

Mark Watson, CTO, ComplyAdvantage

Culture Eats Roadmaps: Kaizen, Education, And Metrics

Technology alone doesn’t deliver outcomes.

Mark emphasises engineering culture: continuous improvement, internal writing to teach and clarify, dedicated learning budgets, and time to share knowledge.

Small, frequent releases beat grand rewrites because they keep risk contained and learning tight.

That approach aligns with well-known software performance research that links measurement and feedback to faster, safer delivery.

Human In The Loop Isn’t Optional

For higher-risk decisions, a human remains the accountable decision-maker.

That's not a blocker to automation; it’s a design constraint.

Successful teams decide up front which steps must be supervised, how to present evidence to reviewers, and what to log so an external party can follow the reasoning.

Regulatory frameworks in the UK and EU both foreground oversight, record-keeping, and clarity about roles for providers and deployers of high-risk AI systems.

"Until models can reliably write and verify their own code to solve novel problems, you won’t get the autonomy some people predict.

Trust, but verify."

Mark Watson, CTO, ComplyAdvantage

Data Is The Differentiator

Generative and agentic techniques won’t help if your system has no context about the customer, the transaction, or the wider network of risk indicators.

Mark’s team invests heavily in data foundations, building connections that increase the surface area for detection and reduce investigative toil.

As those knowledge graphs deepen, the multiple of inferred facts grows, strengthening the “reasoning substrate” that future automation depends on.

That aligns with guidance that stresses data governance and quality as prerequisites for AI effectiveness and fairness.

Getting Started: A Practical Playbook

Start From The End: What Will The Report Need To Prove?

Define the artefacts a regulator or auditor will expect: data lineage, controls applied, reviewer notes, and rationale for the outcome.

Work backwards to design your workflow, logging, and reviewer experience so those proofs fall out naturally.

This reduces “documentation debt” and keeps explainability front-and-centre.

Pick Deterministic Wins First

Focus on high-volume, rules-shaped tasks such as initial case triage, evidence gathering, or narrative drafting for straightforward outcomes.

Automate the handoffs and guardrails before you chase flashier use cases.

Measure the reduction in handling time and rework so you can demonstrate value early.

Instrument, Benchmark, And Re-train On A Cadence

Track model drift, false-positive rates, and reviewer overrides.

Use a fixed cycle for evaluation and updates so performance doesn’t decay silently.

Document the test sets and thresholds you use so changes remain auditable over time.

Design For Vulnerability And Fair Outcomes

Balance efficiency with duty of care, especially where automated flows touch customers directly.

Build escalation paths and clear language for users who need extra support.

Recent guidance in the UK sets expectations for identifying and supporting vulnerable customers consistently.

Align With The Rulebook You’ll Be Measured Against

Map your design to the UK’s data protection guidance on AI and, for EU-facing operations, the AI Act’s deployer obligations.

Clarify roles, oversight, logging, incident reporting, and user information duties early.

This avoids expensive retrofits when frameworks bite.

Interested in further information on AI in compliance and AML?

Watch the CATALYST virtual event: a live stream of ComplyAdvantage's invitation-only London summit where they unveiled how integrated AI is reshaping financial crime risk management for unprecedented efficiency and accelerated business growth.

Watch it now

References

Artificial Intelligence in UK Financial Services - 2024, Bank of England & FCA, 2024

AI Update, Financial Conduct Authority, 2024

AI Live Testing: From Promise To Practice, Financial Conduct Authority, 2025

Guidance on AI and Data Protection, Information Commissioner’s Office, 2023

Explaining Decisions Made With AI, UK Government (GDS), 2025

EU AI Act - Regulatory Framework, European Commission, 2024

AI Act, Article 26 - Obligations of Deployers, European Union (consolidated), 2024

Digital Transformation of AML/CFT - Executive Summary, FATF, 2021

Amending Guidelines on ML/TF Risk Factors, European Banking Authority, 2024

Guidance for the Fair Treatment of Vulnerable Customers, Financial Conduct Authority, 2021

Accelerate State of DevOps (DORA 2024), DORA, 2024

How ComplyAdvantage Uses AI Across The Financial Crime Lifecycle, ComplyAdvantage, 2025

FAQs

What AI use cases deliver value fastest in compliance?

High-volume, rules-shaped steps such as initial alert triage, document summarisation, and evidence gathering usually pay back first.

They’re deterministic, auditable, and relieve pressure on analysts to focus on judgment calls.

How do we balance automation with Consumer Duty expectations?

Design for vulnerable users from the start, create human escalation paths, and test outcomes for fairness as well as accuracy.

Document how the system supports good outcomes, not only operational efficiency.

What should our AI governance pack include?

Scope and risk assessment, data lineage, model cards, validation results, human-oversight design, logging and incident processes, and change history.

Map each element to the relevant UK/EU expectations so reviewers can navigate quickly.

Where are agentic workflows useful today?

Chained tasks with clear interfaces and ground truth, such as gathering KYC evidence, cross-checking watchlists, and drafting case notes for human sign-off.

Keep them narrow, testable, and wrapped in controls.

How do we keep momentum as the tech changes?

Work to a 12-18 month plan with fixed evaluation cycles, so you can adopt safer improvements without derailing delivery.

Measure relentlessly and retire what no longer pulls its weight.

Reviewed by

Sam Kendall, 12.11.2025