A fundamental shift is under way, according to the business analysis firm Gartner: from data exposure to insight exposure. Bart Willemsen, VP Analyst at Gartner, says: โOrganizations have historically focused on protecting raw personal data, but AI [artificial intelligence] can now reconstruct deeply personal insights without ever breaching traditional data controls. Privacy risks are increasingly emerging from what AI algorithms infer about individuals rather than what data is directly exposed.
โInference attacks are particularly dangerous because they often evade conventional detection mechanisms. Individuals can be exposed through AI-generated conclusions rather than leaked records, creating privacy risks that undermine data integrity and are difficult to detect, explain and mitigate.โ
See also a paper from the UK official NCSC (National Cyber Security Centre).
Comment
While Gartnerโs prediction sounds alarming, it wonโt surprise anyone close to this space, says Cynthia Overby, director of strategic security solutions, ZCOE at Rocket Software. She says that AI-driven threats have been steadily increasing in the last year and AI is already inferring sensitive attributes from seemingly harmless information.
She says: โIn addition to the potential for AI tools themselves being exploited, they can assist hackers in significantly reducing the time needed to execute attacks, gather personal information, and now even fill in the gaps based on what information is readily available to them โ without even breaching traditional data controls.
โDue to rapid advances in GenAI and machine learning โ rather than having to manually map an environment or gather personal information โ hackers can now use AI to quickly analyse organisational structures, identify administrators and privileged users, map trust relationships, and make inferences based on this information. This introduces an entirely new threat landscape to personal data, especially considering that the majority of information that these agents need to form inference is also readily available in apps like LinkedIn, X, Instagram, and company websites. For example – inferring financial stress from spending behaviors and comments in widely used apps.
โBuilding resilience in this threat landscape now goes beyond the CISO alone, and requires an entire ecosystem of defence. Protecting personal data is about more than just buying a new tool, itโs about building an entire ecosystem, and understanding how AI systems generate, act on, and use information about individuals to gain access to sensitive data. After all, data is the most valuable asset an organisation has.
โFirstly, a data governance strategy is essential. Knowing where critical data is stored, who can access it and why is key. Organisations should also implement regular configuration and compliance monitoring, as well as vulnerability scanning, addressing weaknesses before they can be exploited. A robust incident response plan and regular testing of disaster recovery processes are therefore critical to minimising downtime and damage in the event of a breach. By proactively prioritising identity management, data privacy and governance, organisations can enhance trust and ultimately reduce AI-driven risk.โ
Downstream violations
Meanwhile according to a Netskope Threat Labs report, based on aggregated usage data collected through the Netskope One platform, downstream data policy violations are now the second most common form of enterprise AI violation, accounting for 924 of every 10,000 alerts, or almost one in ten. Downstream violations happen when an AI service returns information that a user or agent is not authorised to access. Ray Canzanese, Director of Netskope Threat Labs said that the 2026 threat landscape has moved beyond shadow AI discovery into a phase of bi-directional, agentic risk. He said: โWe are no longer just monitoring the prompts employees send to third-party models; we are now governing the integrity of the AI supply chain. The rapid integration of the Model Context Protocol (MCP) effectively bridges our internal data stores with external agents, while the surge in autonomous coding tools like Cursor and Claude Code has drastically lowered the barrier for automated, malicious code execution. For security teams, this mandates a pivot from simple data to bidirectional inspection. We need to treat every agentic interaction as a potential execution vector, not just a data request.โ
