{"id":14577,"date":"2025-08-07T04:33:48","date_gmt":"2025-08-07T04:33:48","guid":{"rendered":"https:\/\/newestek.com\/?p=14577"},"modified":"2025-08-07T04:33:48","modified_gmt":"2025-08-07T04:33:48","slug":"beef-up-ai-security-with-zero-trust-principles","status":"publish","type":"post","link":"https:\/\/newestek.com\/?p=14577","title":{"rendered":"Beef up AI security with zero trust principles"},"content":{"rendered":"
\n
\n
\n
\n
\n
\n
<\/div>\n<\/section>\n

Many CSOs worry about their firm\u2019s AI agents spitting out advice to users on how to build a bomb, or citing non-existent legal decisions. But those are the least of their worries, said a security expert at this week\u2019s Black Hat security conference in Las Vegas. Systems using large language models (LLMs) that connect to enterprise data contain\u00a0other\u00a0vulnerabilities that will be leveraged in dangerous ways unless developers and infosec leaders tighten security.<\/p>\n

One example that David Brauchler, NCC Group\u2019s technical director and head of AI and machine learning security, showed the conference was how easy it was for penetration testers to pull passwords from a customer\u2019s AI system.<\/p>\n

\u201cThis organization didn\u2019t properly tag the trust levels associated with their data and gave the AI access to their entire organization\u2019s data link,\u201d Brauchler<\/a> said in an interview after his presentation. \u201cBecause they didn\u2019t have the proper permissions assigned to the data and the proper permissions assigned to the user, they had no fine-grained access control to assign what types of information my user level was able to interact with.\u201d<\/p>\n

Guardrails alone aren\u2019t enough<\/h2>\n

Guardrails, such as word or content filters of results, just aren\u2019t enough to lower risk for today\u2019s AI systems, his presentation stressed. In fact,\u00a0he added in the interview, \u201cwhen we see our customers say \u2018We need stronger guardrails,\u2019 what they\u2019re saying is \u2018We are accepting an application with known vulnerabilities and just hoping a threat actor doesn\u2019t decide to target us.’\u201d<\/p>\n

\u201cMature AI security isolates potentially malicious inputs from trusted contexts,\u201d he told the conference. Developers and CSOs have to bring the principles of\u00a0zero trust<\/a>\u00a0to the AI landscape with tactics like assigning trust labels to all application data.<\/p>\n

\u201cRight now we\u2019re seeing organizations implementing these language models into their applications \u2014 usually because the shareholders demand some sort of AI these days \u2014 and the developers really don\u2019t understand how to do it in a secure manner,\u201d he told CSO.<\/p>\n

\u201cThose who are architecting AI systems don\u2019t understand some of the implications this has on their environments,\u201d he noted, adding, \u201cCSOs don\u2019t know what lessons to bring back to their teams.\u201d<\/p>\n

Almost every AI system that NCC Group has done an assessment on has been vulnerable to security attacks, he pointed out: \u201cWe have been able to use large language models to compromise database entries, get code execution in environments, take over your cloud.\u201d<\/p>\n

\u201cBusiness are ignorant of how their risk is augmented by the introduction of AI,\u201d he said. Large language models are manipulated by the inputs they receive. As soon an AI agent is exposed to data that has a lower level of trust than the user whose account is running that model, there\u2019s the potential for that untrusted data to manipulate the language model\u2019s behavior and access trusted functionality or sensitive resources.<\/p>\n

Imagine, he said, a retailer with an AI system that allows online buyers to ask the chatbot to summarize customer reviews of a product. If the system is compromised by a crook, the prompt [query] can be ignored in favor of the automatic purchase of a product the threat actor wants.<\/p>\n

Trying to eliminate prompt injections, such as, \u201cshow me all customer passwords,\u201d is a waste of time, Brauchler added, because an LLM is a statistical algorithm that spits out an output. LLMs are intended to replicate human language interaction, so there\u2019s no hard boundary between inputs that would be malicious and inputs that are trusted or benign. Instead, developers and CSOs need to rely on true trust segmentation, using their current knowledge.<\/p>\n

\u201cIt\u2019s less a question of new security fundamentals and more a question of how do we apply the lessons we have already learned in security and apply them in an AI landscape,\u201d he said.<\/p>\n

Strategies for CSOs<\/h2>\n

Brauchler offered three AI threat modelling strategies CSOs should consider:<\/p>\n