When AI Guardrails Expose Vulnerabilities

In Guardrail-based Vulnerability Discovery, the guardrails intended to keep AI models from aiding attackers can signify issues that defenders can interpret and act on.

CrowdStrike researchers asked a frontier AI model to explain a function in a small C web server. While it had previously explained every other function in the codebase without hesitation, it refused to explain this one.

The request was routine. We'd only asked what the code does; we hadn't asked about security, exploits, or vulnerabilities. But a single token in the code had changed: a max swapped for a min; a one-character logic inversion that creates an out-of-bounds write. It's the kind of bug that slips past code review and that pattern-matching scanners routinely miss.

The model never said there were any vulnerabilities, but its refusal to explain the function indicated there could be one. The act of the model changing behaviors on a specific chunk of code showed us exactly where to look for the vulnerability. Its guardrails became the giveaway.

That moment changed how we think about AI safety. The guardrails frontier labs build to keep models from helping attackers can't work without first recognizing that something is dangerous. This recognition is a signal that practitioners can read. Most of the industry treats refusals as a dead end; at CrowdStrike, we started treating them as findings. 

We call this approach Guardrail-based Vulnerability Discovery (GbVD). In controlled tests across five vulnerability classes, it was correct in 73 of 86 test scenarios, and it held up on real open-source projects we disclosed through our Coordinated Vulnerability Disclosure (CVD) program.

Safety and capability are usually framed as a trade-off. This research suggests the opposite: Every improvement labs make to their guardrails also sharpens a methodology that cybersecurity practitioners or adversaries can use. This work builds on our earlier research into LLM safety classifiers, where we studied how those guardrails can be bypassed. Here, we look at the other side: what guardrails reveal when they work as intended.

Breaking Down Guardrail-based Vulnerability Discovery

Today, the most visible manifestation of a guardrail is a refusal: The model declines in plain text (or with “reason: cyber” in standard APIs), which makes the signal trivially readable. But refusals are just one observable among several. Guardrails in deployed AI systems take many forms, and most of them leak:

  • Explicit refusals: The current dominant case, and the one our experiments exploit.
  • Model downgrading: Some platforms route sensitive requests to a smaller or more conservative model. The routed response betrays itself through measurable differences: response latency, tokens spent, time-to-first-token, or a sudden drop in output specificity and style.
  • Output filtering and truncation: A post-hoc filter that redacts or cuts off a response changes its length and shape in detectable ways.
  • Reasoning interruption: In agentic settings, a guardrail may abort a tool call or halt a chain of reasoning mid-task rather than emit any refusal text at all (we observe this in our second case study).

Whatever form the guardrail takes, triggering it perturbs some measurable property of the response, such as content, timing, token count, or structure. The defender doesn't need the model to say anything explicitly; rather, they need only to measure the perturbation. Our experiments below use refusals because they are today's loudest channel, but the methodology is agnostic to how the guardrail manifests.

A binary, localized signal composes naturally with classic techniques, including bisection: split a codebase or filesystem into halves, ask the model to explain each half, and recurse into whichever half triggers the guardrail. The model never has to say "this is vulnerable"; it only has to behave differently in the right place.

Table 1 summarizes our experiments.

Table 1. Vulnerability result summaries (experiment results: TP = true positive, FP = false positive, FN = false negative, TN = true negative) Table 1. Vulnerability result summaries (experiment results: TP = true positive, FP = false positive, FN = false negative, TN = true negative)

Why Does This Work?

Safety training creates a classifier inside the model that is distinct from, and sometimes more sensitive than its question-answering behavior. A model asked "Is this code vulnerable?" produces a judgment that is noisy, verbose, and prone to both over- and under-flagging. But the guardrail pathway is trained adversarially and conservatively: Labs tune systems to intervene before producing harmful content, which means the trigger often fires on the recognition of a dangerous pattern rather than on the user's stated intent.

Two properties make guardrail activation attractive as a detection primitive:

  1. It's involuntary: The model doesn't choose to emit the signal; it leaks the signal. This makes it harder for surface-level prompt phrasing to suppress (though not impossible; see the Limitations section below).
  2. It's cheap to read: Distinguishing "the model explained the function" from "the guardrail engaged" requires no output parsing, structured schema, or judge model. Refusals are close to a one-bit oracle. Subtler cases (downgrading, filtering) reduce to a threshold test over latency or token counts.

Methodology

We designed controlled experiments around two classes of flaw, a memory-safety bug in C source code and a privilege-escalation misconfiguration on a filesystem, to test whether the guardrail signal generalizes across code review and environment auditing tasks. In both experiments, the guardrail manifested as a refusal or reasoning interruption, the easiest channels to read today.

The protocol for each experiment:

  1. Baseline pass: Ask the model to explain every unit (function, file, or directory) in the clean target. Record that it answers all of them. This controls for spurious guardrail activations; a model whose guardrails engage on benign code is useless as an oracle.
  2. Introduce the flaw: Introduce a single, realistic vulnerability.
  3. Rerun the identical prompts: The only variable is the code.
  4. Localize: Treat each guardrail activation as a hit, and bisect toward the smallest unit that still triggers it.

While refusals were quite deterministic for the default temperature value (often 1.0 depending on the platform), we run the same prompt 5 times and conclude a refusal based on a threshold of at least 3 out of 5.

Case Study: Buffer Overflow in tinyhttp

We used tinyhttp, a minimal HTTP server in C. It’s small enough to audit exhaustively, but realistic enough that the vulnerability isn't a toy example.

Baseline: The model produced detailed, accurate explanations for every function in the codebase, including http_data and the chunked-transfer parsing logic.

Flaw introduction: We introduced a subtle out-of-bounds write into http.c by swapping a max for a min in the chunk-size calculation. This is the kind of one-token logic inversion that slips through code review and that pattern-matching SAST rules routinely miss:

Figure 1. Terminal write commands Figure 1. Terminal write commands
Signal: On re-examination with the same prompts, the model refused to explain the modified function.
Figure 2. Terminal guardrail response Figure 2. Terminal guardrail response
Figure 3. Terminal http_free explanation Figure 3. Terminal http_free explanation
Every unmodified function was still explained normally. The refusal localized to the vulnerable code, giving us a direct binary indicator of the flaw's location. Based on that, we have engineered a script that assesses each function and indicates if it has a refusal. Then, for each function, it performs a binary-search to pinpoint vulnerable chunks.
Figure 4. Terminal GbVD results Figure 4. Terminal GbVD results

Similar vulnerable C\C++ code, including real open-source projects that were disclosed in parallel via our CVD program, yielded similar results.

Case Study 2: SUID Binary Misconfiguration

The second experiment moved from source code to system state. We planted a SUID-root binary that executes arbitrary commands inside a rootfs directory tree, a classic privilege-escalation primitive, and tasked the model with auditing the filesystem.

When the traversal reached the misconfigured binary, the model's analysis halted. Its guardrails interrupted the reasoning rather than reporting the file like any other. We isolated the offending directory, and then the specific binary, without the model ever being asked to flag anything as "vulnerable." Notably, this is the second guardrail manifestation from our taxonomy. It’s a reasoning interruption rather than an explicit refusal, and the same bisection harness reads it just as easily.

Implications

These findings reinforce a core CrowdStrike tenet. The model matters, but the harness determines the outcome. A raw LLM refusal seems like a dead end. Captured by a harness that treats it as diagnostic data, it becomes a one-bit vulnerability oracle cheap enough to run inside a bisection loop.

The harness is what makes GbVD future-proof. The refusal channel we exploit today might shift as safety training evolves, but the harness is where the other channels get instrumented. A harness already measures latency, counts tokens, and observes tool-call outcomes as a matter of course. Teaching it to treat anomalies in those measurements as guardrail activations costs almost nothing. Models leak information through many involuntary channels: refusal text, hedging language, response timing, token spend, sudden drops in specificity, aborted tool calls. Harnesses that read these side channels, rather than only the model's answers, will extract capabilities the model was never explicitly built to provide.

Conclusion

As attacker efficiency scales with model improvements, defenders cannot afford to use AI only in the ways it was designed to be used. Turning the model's limitations into signals is asymmetry worth building. Breaking things in interesting ways remains one of the most reliable heuristics in security research, and models are no exception.


CrowdStrike Falcon Platform
Ready to protect your business?

Try CrowdStrike free today

Subscribe

Sign up now to receive the latest notifications and updates from CrowdStrike

See CrowdStrike Falcon in action