AI bug hunters: finding is not fixing

Research cutoff: 1 November 2026. Announcements and access conditions can change.

§1 · The answer

Promising hunters. An unproven cure for the patch gap.

Security AI is already useful for discovery and candidate repairs. Evidence that it closes the gap across deployed systems is not yet there.

Google reports 55 unique confirmed V8 issues in a fixed-invocation test of Gemini 3.5 Flash Cyber, compared with 47 and 36 for two other models in the same experiment. A separate Project Zero study counted 45 actionable BigSleep reports versus 37 for the Fuzzilli fuzzer in V8. Neither comparison proves that organizations deploy fixes faster; the experiments have different designs and should not be combined into a model leaderboard.

Google also reported 72 CodeMender security fixes upstreamed over six months in an earlier generation of the system, with human review. Upstream acceptance is meaningful progress—but it is not proof every downstream user updated. The key outcome is verified remediation on exposed assets, not the number of convincing reports.

§2 · The capability

A model is not the whole bug hunter

A model reasons; an agent gives it a workspace, tools, tests, and a loop. The result belongs to the whole setup.

Flash Cyber is deployed in Google's CodeMender agent, which can inspect code, run tools and validate candidate vulnerabilities. In its CyberGym comparison Google allowed up to five model calls per final report. The V8 comparison below is a different fixed-invocation experiment. Treat both as vendor-reported research results, not interchangeable product guarantees.

In that V8 test, Google says 10 issues found by Flash Cyber were missed by both comparison models. That suggests complementary search paths, not a guaranteed independent advantage on every codebase. Confirmed issues are not necessarily CVEs, accepted patches, or deployed fixes.

Discovery and validation are separate jobs

A useful agent can trace data flow, develop a reproducible test in an isolated environment, and propose a fix. But it can also produce a plausible false alarm or a patch that breaks behavior. Human reviewers must check the evidence, tests, compatibility and disclosure path. In Google's OSS-Fuzz AI patch beta, human patch shepherds review eligible proposed fixes, and projects can opt out.

§3 · The strongest evidence—and its limits

Complement, not replacement

A conventional fuzzer and an AI agent do not see exactly the same bugs. That difference can be valuable even when neither is a complete answer.

Project Zero's April–September 2026 V8 comparison counted 45 actionable vulnerability reports for BigSleep and 37 for Fuzzilli. Their bugtracker records overlapped on 20 of 81 actionable bugs; the researcher warns that this overlap likely understates shared coverage. A separate historical re-discovery experiment found roughly 60% overlap, using a different cohort and method. The studies are not contradictory and their counts must not be added.

The 2025 DARPA AI Cyber Challenge reported that finalist systems found 54 of 63 synthetic challenge vulnerabilities. That is impressive within a controlled competition, not a measured share of real-world vulnerabilities or evidence of installed patches.

Even an upstream fix can stop short of end-user protection: a PHP response involved independent discovery, maintainers, coordinated disclosure and an upstream release. Whether each affected estate installed the fix remains a separate question. Read the PHP2Shell case →

§4 · Dual use and access

The gate solves one problem—and creates another

The same reasoning that validates an authorized weakness can aid misuse. Verification may reduce risk, but it can also exclude legitimate defenders.

The UK's AI Security Institute evaluated an early checkpoint of general GPT-5.5 on simulated cyber tasks; one multi-step range was completed in two of ten attempts, without real defenders in the environment. This is evidence of dual-use capability in a controlled setup, not a forecast of attack success in the wild or a test of GPT-5.5-Cyber's access policy.

OpenAI's April 2026 trusted-access announcement combined an ambition for broader defensive access with verification and a restricted cyber tier. On 1 May, AI News Digest called the GPT-5.5 Cyber rollout “quietly” gated. The archive exposes the headline and brief, not the full issue; “quietly” is the newsletter's characterization. On 7 May, OpenAI publicly described the limited preview and its controls. That later explanation does not establish what had been disclosed before the newsletter appeared.

OpenAI said its initial GPT-5.5-Cyber preview was trained primarily to be more permissive for specialized authorized work, not intended to significantly increase capability over base GPT-5.5. Fewer refusals are not, by themselves, proof of better accuracy. Google's July pilot similarly reserved Flash Cyber for governments and trusted partners. These are time-stamped announcements, not a current eligibility directory.

What would settle the access argument?

Publish the number of applicants, approvals, denials, appeal outcomes and wait times by defender type, with privacy-preserving categories. Pair that with independently evaluated misuse incidents and downstream defensive outcomes. We found no representative, controlled comparison that resolves the net effect of gating. A small maintainer and a large critical-infrastructure operator may face very different costs of exclusion.

§5 · A defender's starting plan

Start with a bounded experiment, not an autonomous patch bot

Use what you own, measure what you can verify, and keep a human in the merge path.

1. Choose a permitted target. Select an owned repository and named owner; document authorization, scope and disclosure rules. Never test third-party production systems without permission.
2. Establish a baseline. Record current time from validated finding to verified deployment, reviewer hours, false-positive burden and regression rate. Inventory affected assets before judging success.
3. Use open tooling first. Explore OSS-Fuzz and ClusterFuzzLite or the AIxCC open-source systems. OSS-Fuzz's hosted patch beta has project, language and bug-class eligibility rules. Open code does not make compute, model access or maintenance free.
4. Contain execution and data. Use isolated test environments with no production credentials, restricted or disabled network access, least privilege and resource limits. Review code and logs for secrets before using any external model. Select a model appropriate to your own data policy and budget.
5. Validate before merge. Reproduce safely, run regression and security tests, have a human review the diff, and coordinate disclosure with maintainers. Do not equate generated patches with verified fixes.
6. Measure the last mile. Ship through ordinary change controls, verify installation on the affected assets, watch for regressions, and retain a rollback. Use EPSS as one prioritization signal, not as evidence of remediation.

Compare a matched or randomized set against existing workflows when feasible. Stop or narrow the pilot if review burden, unsafe changes, secret exposure or regressions outweigh verified remediation gains. This is a recommended experiment, not a claim that one has already succeeded.

§6 · The bottleneck

If deployment stalls, discovery alone cannot close the gap

Finding more real issues can help—but it can also build a larger queue if review and rollout do not keep up.

Compare the evidence by what it actually measures, not by whether the headline sounds impressive.

§7 · What would change the verdict

Six questions the next evidence must answer

These are proposed measurements, not observations. Missing means the public record does not support a current reading.

§8 · The people in the loop

The patch gap has more than one owner

An AI agent may start the work; maintainers and operators decide whether protection arrives.