There are clearly huge risks to our national security and society if the right guardrails are not put in place, First Secretary of State Louise Haigh told the Trade Union Congress (TUC) in Brighton in a speech on September 15. She said: “We have a solemn duty to mitigate the risks and ensure that itโs for the benefit – not the detriment – of our society.”
Even before these recent concerns were raised regarding the possibility of human extinction, the wave of rogue AI agent attacks already demonstrated that frontier developers are moving faster than they should, with arbitrary third parties ending up as the victims of AI’s actions, according to Oliver Simonnet, Lead Cybersecurity Researcher at CultureAI. He said: “It could just as easily be a case where the company affected is healthcare, energy providers, or finance, which could then easily affect millions of lives. Then there is also the societal impact as AI use errodes problem solving and critical thinking skills in established professionals and new generations. Lastly, energy requirements also need addressing to reduce AI impact on the planet and ensure we can sustain its advancements. These are all solvable problems at the right pacing, but not if we simply storm ahead towards the next advancement.
AI kill switch
He added that an “AI kill switch” is something to explore. He said: “Even with smaller scale rogue agent incidents, having a localised kill switch could help rapidly contain the fallout before it spread, especially as AI operates much faster than humans can possibly hope to react. Features like this have been integrated into malware for years, so Im sure there is some way it can be integrated into AI tech. Though, I don’t think it could be designed as a broad “turn off all AI completely” switch, as AI is now far too integrated into all aspects of technology and society, and our own defensive capability now depends on it. Turning it “all” off could cause unknown damage across all sectors, from healthcare to critical infrastructure, and in the worst case potentially result in the loss of life.”
Work alongside physical AI
Meanwhile the market analysts Gartner at their IT Symposium/Xpo predicted that by 2030, some 80 percent of frontline workers will work alongside physical AI. While Gartnerโs prediction seems extreme, increasingly capable AI orchestration and widespread investment has meant that AI was always going to reshape the workforce, commented Iain Davidson, head of product marketing, Wireless Logic. He said: “This comes as organisations look to lower operating costs, boost productivity and gain deeper insights that contribute directly to measurable business outcomes. But as AI moves from the digital world into the physical, its success is entirely dependent on the infrastructure that supports it behind the scenes.
โPhysical AI will generate and harness vast amounts of data that businesses will increasingly come to rely on for operational efficiency. As we see deployments increase leading up to 2030, the resilience of the IoT will be critical to keep physical AI systems connected and transmitting, not least because these systems are likely to underpin services in frontline industries such as healthcare, manufacturing and transport.
โAs highlighted by Gartner, organisations must adopt a safety-first approach when implementing physical AI and invest in scalable platforms. Resilient IoT that prioritises enhanced security and minimises downtime will be central to the foundation of physical AI. Organisations should not simply rush to deploy smarter robots and models โ they need systems and processes in place to ensure their connectivity is resilient, and that physical AIsystems can be monitored, provisioned and ultimately execute tasks safely in real-time.โ
Zero trust principles
As organisations continue to expand across cloud, third-party platforms and AI-powered services, the traditional concept of trust based on network location is becoming increasingly obsolete. Security is no longer defined by whether a user or system operates within a corporate boundary, but by the ability to verify every access request, validate every interaction, and apply controls that are proportionate to the risk. More from this discussion between Sreenivasulu Gajula FBCS, Lead Principal Full-Stack Engineer with Fidelity, with Martin Cooper on applying zero trust principles to AI and cloud, on the BCS (Chartered Institute for IT) website.
