-optimized.jpg){kind=avatar}
A data centre of genius fraudsters is the next deepfake wave, says Ricardo Amper, CEO at Incode.
UK Government figures show an estimated eight million deepfakes were shared in 2025, up from 500,000 in 2023, a 16-fold increase in two years. And now, as the nature of deepfakes continues to evolve, we are reaching a point where even audio and video can no longer be treated as evidence by default.
In many organisations, users have already learned to treat unexpected email as potentially hostile, not because every message is a phishing attempt, but because it only takes one convincing email to cause serious damage to an entire business. Deepfakes are now pushing us towards the same posture for phone calls, voice notes and video meetings. The difference is psychological. Email has always felt indirect. Video and voice feel human. They feel like proof.
The synthetic media that captures headlines – celebrity impersonations, political clips, novelty content – is only the visible edge of a deeper shift. In real fraud investigations across banks, consumer platforms and telecom providers, what is emerging is not entertainment-grade manipulation. It is operational, targeted and engineered for return on investment. The inflection point comes when deepfakes stop being a standalone trick and become a capability embedded inside autonomous AI agents.
Dario Amodei described the promise of advanced AI agents as a data centre of geniuses working in parallel. It is an inspiring description of productivity at scale. But it also forces a harder question: what happens when criminal networks gain access to their own version of that infrastructure?
Scaling effect: precision at industrial volume
Historically, social engineering has been limited by human capacity. Fraud rings cannot instantly recruit thousands of highly skilled operators with the language fluency, cultural nuance and business context required to convincingly impersonate senior executives. Even sophisticated attackers are constrained by fatigue, inconsistency and the cognitive load of managing multiple tailored conversations at once.
Autonomous agents remove those constraints. They do not simply generate a convincing face or clone a voice; they construct a coherent persona. They can replicate how a CFO speaks and writes, mirroring cadence, punctuation, preferred phrasing and subtle expressions of urgency. They can reference live projects, internal shorthand and vendor relationships with ease.
The real weapon is context. A modern employee’s digital footprint includes collaboration threads, document repositories, calendar invites, vendor exchanges and deal summaries. If an AI-driven attacker gains visibility into even a fraction of that ecosystem, it can model the organisation’s rhythms faster and more comprehensively than any human adversary. The attack then shifts from something crude – “Transfer £5 million now” – to something almost indistinguishable from legitimate communication, tied to a specific acquisition, referencing the correct entity name and aligned with an upcoming board update.
This is where scale changes everything. It will not be one attempt, but thousands running in parallel, refining tactics in real time. Always on. Always adapting. Fraud becomes infrastructure rather than activity; precision delivered at industrial volume.
Detection alone will fail
The traditional response to synthetic media has focused on detection: spot the visual artefact, notice the unnatural blink, listen for distortion in the audio. Train staff to be vigilant. That strategy assumes imperfections will remain obvious.
In practice, the most dangerous attacks will not look flawed. A synthetic video call from a CEO requesting urgent approval will arrive at a plausible moment, reference accurate internal details and align with ongoing business priorities. The face will look right. The voice will sound right. The context will be right. When everything appears authentic, human intuition becomes unreliable.
Telling employees to trust their instincts places the burden on the least reliable layer of defence. As AI agents become more capable, detection based solely on artefacts will struggle to keep pace. Static rules and signature-based systems are poorly matched against adaptive systems that continuously evolve.
The more sustainable response is structural. Organisations must redesign trust rather than simply sharpening suspicion. Audio and video should be treated as untrusted inputs when high-risk actions are involved. No single channel, however convincing, should be sufficient to authorise significant money movement, account recovery, access changes or privileged operations.
Trust for the agent era
Redesigning trust means embedding strong identity verification directly into sensitive workflows. High-value decisions should require confirmation through verified, independent channels rather than approval granted in the moment on a call. Advanced biometric liveness, device binding and behavioural monitoring need to become standard components of critical processes, not optional safeguards layered on afterwards.
It also means fighting AI with AI. Defence must become behavioural and continuous, analysing patterns across systems and identifying anomalies in real time. The objective is not to determine whether a single video looks fake, but whether the overall interaction aligns with expected patterns of behaviour, risk and authority.
We are moving from a world in which identity is presumed authentic unless proven otherwise, to one in which identity must be continuously verified in context. That shift is not paranoia; it is proportional adaptation. AI will drive extraordinary productivity and innovation. It will also equip adversaries with tools once associated with state-level capability.
A data centre of geniuses can accelerate growth and creativity and exploit poorly designed trust models just as efficiently. The question for security leaders is not whether deepfakes will improve, because they will, but whether systems of verification and authorisation will evolve at the same pace. In the agent era, resilience will depend less on spotting the fake and more on ensuring that even a perfect imitation cannot, on its own, unlock money, access or identity.
Photo by Mark Rowe: street art.
