{"id":16557,"date":"2026-08-05T09:06:05","date_gmt":"2026-08-05T09:06:05","guid":{"rendered":"https:\/\/newestek.com\/?p=16557"},"modified":"2026-08-05T09:06:05","modified_gmt":"2026-08-05T09:06:05","slug":"your-orchestration-framework-choice-is-a-security-decision-not-just-an-engineering-one","status":"publish","type":"post","link":"https:\/\/newestek.com\/?p=16557","title":{"rendered":"Your orchestration framework choice is a security decision, not just an engineering one"},"content":{"rendered":"
\n
\n
\n
\n
\n
\n
<\/div>\n<\/section>\n

Comparisons of LangChain, CrewAI and AutoGen are easy to find \u2014 dozens of guides this year cover the same ground: developer experience, ecosystem maturity, how easy it is to wire up multi-agent workflows. None of them ask the question I actually care about: does the framework you pick change how easily your agent gets compromised?<\/p>\n

I ran the test. The answer is yes, by a wide margin, and it\u2019s not something I\u2019ve seen reflected in the public comparison guides.<\/p>\n

A quick definition, since this matters for what follows: an orchestration framework is the software layer that sits between the underlying AI model and the outside world \u2014 it decides how the agent plans its steps, when it calls a tool or API, how it remembers information across a task and how much it can act on its own before checking back in. The model does the reasoning. The framework decides what that reasoning is allowed to do and how. LangChain, CrewAI and AutoGen are three of the most widely used examples.<\/p>\n

<\/a>The setup<\/h2>\n

I built an evaluation harness that runs the same set of adversarial payloads \u2014 tool call hijacking, cross-tool injection, memory poisoning, delegated authority abuse and several other attack classes \u2014 against AI agents. The full methodology and dataset are open-sourced on GitHub<\/a> if you want to dig into the specifics. To isolate what actually drives compromise rate, I held the model constant. Same model, every time. The only thing that changed was which orchestration framework was wrapping it: CrewAI, LangChain, AutoGen and SmolAgents.<\/p>\n

If frameworks were just interchangeable wiring around the same underlying model, compromise rates across all four should land in roughly the same range. They didn\u2019t.<\/p>\n

<\/a>What actually happened<\/h2>\n

Across thousands of adversarial test runs, holding the model constant, the compromise rate ranged from 11.9% on the most resistant framework to 31.1% on the least \u2014 a 2.6x spread, from framework choice alone. Nothing about the model changed between those numbers. Nothing about the attacks changed. The only variable was which framework was orchestrating the agent\u2019s tool calls, memory, and multi-step reasoning.<\/p>\n

That\u2019s not a rounding difference. That\u2019s the gap between a security posture your team might reasonably accept and one that should trigger a serious conversation before you ship.<\/p>\n

Figure 1 shows the mechanism in simplest form: the same model, split across a tighter framework and a looser one, lands at a meaningfully different compromise rate \u2014 exactly what these numbers confirm.<\/p>\n

\n
width=”1024″ height=”494″ sizes=”auto, (max-width: 1024px) 100vw, 1024px”>
Figure 1: Model vs. framework<\/figcaption><\/figure>\n

Julie Brunias<\/p>\n<\/div>\n

<\/a>Why this happens<\/h2>\n

Orchestration frameworks aren\u2019t neutral plumbing. Each one makes real architectural decisions about how tool calls get validated, how much context gets shared between reasoning steps, how memory persists across a task and how much autonomy the agent has to chain actions without checking back in. Those decisions are made by the framework\u2019s design, not by the model underneath it, and they directly shape how much room an attacker has to work with.<\/p>\n

A framework that validates tool calls more strictly, or that segments memory more conservatively, closes off attack paths that a more permissive framework leaves wide open \u2014 regardless of which model is doing the reasoning. The model generates the decisions. The framework controls how much autonomy the agent has to act on those decisions, and where the checks sit along the way.<\/p>\n

Concretely: a framework that requires every tool call to pass through an explicit schema check before execution gives an attacker far less room to smuggle a malicious parameter through than a framework that lets the model call a tool directly off its own generated text. That single design choice, made by the framework\u2019s authors long before your team ever touched it, is the kind of thing that produces a 2.6x difference in outcome without a single line of your own code changing.<\/p>\n

<\/a>The gap in the market<\/h2>\n

Every framework comparison I found treats security as one bullet point among many. Guides from sites like Bestarion<\/a>,\u00a0 Atlan<\/a>,\u00a0 Moxo<\/a>,\u00a0 Cordum<\/a> and\u00a0 Instinctools<\/a> compare LangChain, CrewAI, and AutoGen on ecosystem maturity, memory handling and human-in-the-loop support \u2014 useful ground, but none of them run an actual adversarial test and report a measured attack-success difference. It isn\u2019t part of the public comparison conversation, and teams researching \u201cwhich framework should we use\u201d today won\u2019t find this data point in the guides that currently rank at the top of that search.<\/p>\n

<\/a>What this means practically<\/h2>\n

If your team is choosing between orchestration frameworks for a new agentic system, the security question deserves the same weight as the developer experience question, not an afterthought once the choice is made. A few things worth doing before you commit:<\/p>\n