Skip to main content
Mark Watson CTO Comply Advantage
8 min

Beyond the Hype: Practical Lessons on Building With AI in Regulated Industries

Posted by Picture of Sam Kendall Sam Kendall

AI promises the earth, but only disciplined teams turn it into dependable value in regulated spaces.

Mark Watson, CTO at ComplyAdvantage, builds AI-driven tools that help financial institutions detect and prevent financial crime.

You can watch this video on YouTube or listen to the interview on our podcast channel, or listen on Apple Podcasts.

Regulated firms need more than model performance. They need measurement, explainability, and culture that turn AI experiments into workflows that stand up to supervision and make life easier for customers.

From false-positive triage to auditable automation, the practical question is how to move beyond hype towards systems teams can defend when scrutiny arrives.

Created from episode transcript

The Shift From Machine Learning To Practical AI

Modern language models changed what compliance teams can do with unstructured text, which sits at the heart of financial crime detection and adverse media screening.

That shift supports faster iteration, simpler retraining cycles, and better results on real-world data than many older, hand-tuned approaches.

Recent UK market analysis points to broad adoption across financial services, with regulators emphasising governance, validation, and accountability alongside innovation.

Where AI Adoption Stands In UK Financial Services

The Bank of England and FCA survey points to broad adoption across financial services, with supervisors emphasising governance, validation, and accountability alongside innovation.

That context helps teams set expectations, secure buy-in, and align AI projects with emerging rules and supervisory focus.

Inside The Product: Where AI Works Today

At ComplyAdvantage, AI shows up across the lifecycle: inside customer-facing products, within engineering, and for internal productivity.

In adverse media analysis, older NLP models might take weeks to retrain; newer approaches can be updated in hours and at far lower operational cost.

That allows faster responses to new typologies without waiting for long modelling cycles.

Incremental improvements compound when teams verify and measure them relentlessly, rather than waiting for a perfect model before shipping.

How ComplyAdvantage uses AI offers a useful snapshot of how those applications show up in day-to-day financial crime workflows.

Automating What Should Be Automated

In compliance operations, the biggest wins often come from triaging noise rather than chasing novelty.

High-sensitivity systems generate false positives that must be reviewed, explained, and either released or escalated.

AI can now handle a meaningful slice of that workload with deterministic, auditable steps, cutting queue lengths and time to resolution.

The net effect is fewer unnecessary holds for legitimate customers and more capacity for analysts to focus on the edge cases that really matter.

Sector guidance recognises this direction of travel, highlighting where digital transformation can strengthen AML effectiveness when deployed responsibly.

The EBA's updated ML/TF risk factor guidance also reinforces the need for firms to keep risk assessments current as typologies and delivery models change.

Move Fast, But Only As Fast As The Tech Allows

The practical rule is simple: run at the speed the technology can safely support.

Autonomous agents should not be treated as self-improving beyond narrow, well-scoped tasks today.

Instead, stitch together clear, deterministic workflows, add human oversight where judgement or explainability is required, and measure everything.

That mindset mirrors supervisory updates that stress testing, validation, and explainability as non-negotiables for AI in financial services.

The FCA's work on AI live testing shows the same direction in practice: innovation is welcome, but only where governance keeps pace with deployment.

Design For Explainability From Day One

In regulated industries, decisions must be reconstructed and explained long after they are made.

Teams that scatter logic across multiple tools often discover too late that the evidence is fragmented, inconsistent, or hard to reconcile.

A single-platform approach, where possible, with consistent data foundations, immutable audit trails, and a common reasoning layer makes that reconstruction easier.

When a regulator asks why, teams need to show inputs, transformations, and outcomes in one coherent line of sight.

Guidance on explaining AI-assisted outcomes reinforces the need for accessible, human-centred explanations alongside technical logs.

"Don't get over your skis. Automate the deterministic steps, instrument everything, and keep humans in the loop where the obligation can't be fully satisfied by software."

Mark Watson, CTO, ComplyAdvantage

That discipline becomes harder to ignore as AI touches more customer-facing and compliance workflows.

"When AI starts touching compliance workflows, auditability stops being a technical detail. Teams need evidence of what was checked, who reviewed it, and what happened if a decision is challenged later."

Michael Wakefield, CTO, Beyond Encryption (Mailock)

That evidence question sits alongside the delivery habits that keep AI programmes moving without losing control.

Culture Eats Roadmaps: Kaizen, Education, And Metrics

Technology alone does not deliver outcomes.

Strong engineering culture - continuous improvement, internal writing to teach and clarify, dedicated learning budgets, and time to share knowledge - keeps delivery grounded in learning rather than hype.

Small, frequent releases beat grand rewrites because they keep risk contained and learning tight.

That approach aligns with well-known software performance research that links measurement and feedback to faster, safer delivery.

Human In The Loop Is A Design Requirement

For higher-risk decisions, a human remains the accountable decision-maker.

Automation still needs explicit design for supervision: which steps must be reviewed, how evidence is presented to reviewers, and what gets logged so an external party can follow the reasoning.

Regulatory frameworks in the UK and EU both foreground oversight, record-keeping, and clarity about roles for providers and deployers of high-risk AI systems.

The EU AI Act regulatory framework and deployer obligations make those roles explicit for firms operating across borders.

"Until models can reliably write and verify their own code to solve novel problems, you won't get the autonomy some people predict. Trust, but verify."

Mark Watson, CTO, ComplyAdvantage

Until that changes, the useful design choice is to automate deterministic steps and keep accountability with people where judgement is required.

Deterministic automation first. Human judgement where accountability cannot be delegated.

Even with those guardrails in place, the quality of the underlying data still decides how far automation can go.

Data Is The Differentiator

Generative and agentic techniques will not help if a system has no context about the customer, the transaction, or the wider network of risk indicators.

Data foundations matter: connections that increase the surface area for detection and reduce investigative toil.

 

Choosing The Right Customer Channel?

Read our research on portals, logins, email, and post before deciding how customers should receive important documents.

Read the customer preference research

As knowledge graphs deepen, the volume of inferred facts grows, strengthening the reasoning substrate that future automation depends on.

That aligns with guidance that stresses data governance and quality as prerequisites for AI effectiveness and fairness.

The sections below turn those principles into a practical starting playbook for compliance and financial crime teams.

Getting Started: A Practical Playbook

Start From The End: What Will The Report Need To Prove?

Define the artefacts a regulator or auditor will expect: data lineage, controls applied, reviewer notes, and rationale for the outcome.

Work backwards to design your workflow, logging, and reviewer experience so those proofs fall out naturally.

This reduces documentation debt and keeps explainability front and centre.

Pick Deterministic Wins First

Focus on high-volume, rules-shaped tasks such as initial case triage, evidence gathering, or narrative drafting for straightforward outcomes.

Automate the handoffs and guardrails before you chase flashier use cases.

Measure the reduction in handling time and rework so you can demonstrate value early.

Instrument, Benchmark, And Re-train On A Cadence

Track model drift, false-positive rates, and reviewer overrides.

Use a fixed cycle for evaluation and updates so performance does not decay silently.

Document the test sets and thresholds you use so changes remain auditable over time.

Checks Before Scaling AI In Compliance

  • Can you reconstruct inputs, transformations, and outcomes for a challenged decision?
  • Which steps stay human-supervised, and what evidence does the reviewer see?
  • Are false-positive rates, overrides, and drift tracked on a fixed cadence?

Those checks should sit alongside customer-facing safeguards, not replace them.

Design For Vulnerability And Fair Outcomes

Balance efficiency with duty of care, especially where automated flows touch customers directly.

Build escalation paths and clear language for users who need extra support.

Recent guidance in the UK sets expectations for identifying and supporting vulnerable customers consistently.

Align With The Rulebook You Will Be Measured Against

Map your design to the UK's data protection guidance on AI and, for EU-facing operations, the deployer obligations.

Clarify roles, oversight, logging, incident reporting, and user information duties early.

This avoids expensive retrofits when frameworks bite.

Interested in further information on AI in compliance and AML?

Watch the CATALYST virtual event: a live stream of ComplyAdvantage's invitation-only London summit where they unveiled how integrated AI is reshaping financial crime risk management for unprecedented efficiency and accelerated business growth. Watch it now.

 

FAQs

What AI Use Cases Deliver Value Fastest in Compliance?

High-volume, rules-shaped steps such as initial alert triage, document summarisation, and evidence gathering usually pay back first.

They are deterministic, auditable, and relieve pressure on analysts to focus on judgment calls.

How Do We Balance Automation with Consumer Duty Expectations?

Design for vulnerable users from the start, create human escalation paths, and test outcomes for fairness as well as accuracy.

Document how the system supports good outcomes, not only operational efficiency.

What Should Our AI Governance Pack Include?

Scope and risk assessment, data lineage, model cards, validation results, human-oversight design, logging and incident processes, and change history.

Map each element to the relevant UK/EU expectations so reviewers can navigate quickly.

Where Are Agentic Workflows Useful Today?

Chained tasks with clear interfaces and ground truth, such as gathering KYC evidence, cross-checking watchlists, and drafting case notes for human sign-off.

Keep them narrow, testable, and wrapped in controls.

How Do We Keep Momentum as the Tech Changes?

Work to a 12-18 month plan with fixed evaluation cycles, so you can adopt safer improvements without derailing delivery.

Measure relentlessly and retire what no longer pulls its weight.

 

References

Artificial Intelligence in UK Financial Services - 2024, Bank of England & FCA, 2024

AI Update, Financial Conduct Authority, 2024

AI Live Testing: From Promise To Practice, Financial Conduct Authority, 2025

Guidance on AI and Data Protection, Information Commissioner's Office, 2023

Explaining Decisions Made With AI, UK Government (GDS), 2025

EU AI Act - Regulatory Framework, European Commission, 2024

AI Act, Article 26 - Obligations of Deployers, European Union (consolidated), 2024

Digital Transformation of AML/CFT - Executive Summary, FATF, 2021

Amending Guidelines on ML/TF Risk Factors, European Banking Authority, 2024

Guidance for the Fair Treatment of Vulnerable Customers, Financial Conduct Authority, 2021

Accelerate State of DevOps (DORA 2024), DORA, 2024

How ComplyAdvantage Uses AI Across The Financial Crime Lifecycle, ComplyAdvantage, 2025

Mark Watson LinkedIn Profile, LinkedIn

ComplyAdvantage, ComplyAdvantage

CATALYST Virtual Event Registration, ComplyAdvantage

Beyond the Hype: Practical Lessons on Building With AI in Regulated Industries, Mark Watson, ComplyAdvantage (#28), Apple Podcasts, 2025

Reviewed by

Sam Kendall, 29.05.26

This content is for general information only and is not legal advice.

 

Originally posted on 25 11 25
Last updated on June 5, 2026

Posted by:  Sam Kendall

Sam Kendall works on digital marketing at Beyond Encryption, helping build B2B marketing activity around research, first principles, and sustainable growth. He writes about marketing effectiveness, positioning, customer communications, and digital culture, with longer-form work published at ATNL.

Return to listing