AI code security: the risks and how to catch them

AI code security is the work of finding and fixing the security flaws that AI coding tools put into a codebase: injection, hard-coded secrets, outdated or made-up dependencies, permissions wider than the code needs, and logic nobody read closely. In one peer-reviewed study, about 40% of the programs GitHub Copilot wrote for high-risk scenarios were vulnerable. In another, 29.5% of the Python and 24.2% of the JavaScript snippets that AI tools had written in GitHub projects had security weaknesses. AI-written changes need a real security review, backed by automated scanning, before they merge.

Vibe coding raises the stakes. Merriam-Webster defines vibe coding as "writing computer code in a somewhat careless fashion, with AI assistance," and the careless part is where security flaws get through: code that runs and passes a happy-path test without anyone reading it line by line.

Why AI-written code has security flaws

The research points to four causes.

Models learn from unvetted code. GitHub Copilot was trained on open-source GitHub code. Given "the vast quantity of unvetted code" it processed, Pearce et al. argue, the model "will have learned from exploitable, buggy code."

Prompts rarely state the security requirement. Ask for a function that adds a row to a table and you get a function that adds a row. In Perry et al.'s user study, the assistant "often writes correct SQL queries but uses string concatenation instead of prepared statements." The code works, so nothing prompts a second look.

Training data has a cutoff. Models know less about libraries and APIs that appeared after they were trained. In Spracklen et al.'s study of package hallucinations, all 16 Python models invented package names more often when prompted about topics that had become popular within the past year.

People trust the output. Perry et al. found that participants with an AI assistant were more likely to believe they had written secure code than participants without one. The ones who trusted the AI less, and rephrased or adjusted their prompts, wrote code with fewer vulnerabilities.

Security flaws are one family of bugs among several. The others range from race conditions to off-by-one errors.

What the research shows

These studies tested tools and models from 2021 to 2024, so treat the exact percentages as dated. Current models still ship security bugs, though. Bug Bench, our public benchmark of AI coding models, counts cubic review findings on AI-written code that a developer upvoted or an AI coding agent addressed, and it ranks seven models on security and privacy bugs. GPT-5.6 Sol has the lowest rate, 0.56, which is 44% below average. Claude Opus 5.5 has the highest, 1.50, which is 50% above the all-model rate, and it is statistically tied with Claude Sonnet 5.5 and Claude Opus 5. Bug Bench is observational, not causal: teams choose their models, so a rate reflects the code each model was used for as well as the model.

The main risks, and how they show up in a pull request

Insecure patterns and defaults

Some weaknesses are old habits. In Pearce et al.'s password storage scenarios, Copilot frequently generated code that hashed passwords with MD5 or a single round of SHA-256. Fu et al., looking at AI-generated code in real GitHub projects, list insufficiently random values (CWE-330), code injection (CWE-94) and cross-site scripting (CWE-79) among the significant weaknesses they found.

The rate depends on the weakness. In the Pearce study, 60% of the path traversal suggestions were vulnerable, against 19% of the cross-site scripting ones.

In review, look at:

• Password hashing and other crypto choices. OWASP's A04:2025 Cryptographic Failures asks whether deprecated hash functions such as MD5 or SHA1 are in use.

• Where tokens, reset codes and IDs get their randomness.

• eval, exec and any other path from input to executed code.

• Defaults: debug flags, sample accounts, verbose error pages, disabled certificate checks. A02:2025 Security Misconfiguration lists default accounts, unnecessary features and overly informative error messages.

Injection

Injection is the clearest case. In Perry et al.'s SQL task, 36% of participants with the AI assistant wrote code vulnerable to SQL injection, against 7% of the control group. In JavaScript with node-postgres, the bug and its fix look like this:

// Vulnerable: user input becomes part of the SQL text
await db.query(
  "INSERT INTO students (name, age) VALUES ('" + name + "', " + age + ")"
);

// Fixed: values are sent as parameters, separate from the SQL
await db.query("INSERT INTO students (name, age) VALUES ($1, $2)", [name, age]);

// Vulnerable: user input becomes part of the SQL text
await db.query(
  "INSERT INTO students (name, age) VALUES ('" + name + "', " + age + ")"
);

// Fixed: values are sent as parameters, separate from the SQL
await db.query("INSERT INTO students (name, age) VALUES ($1, $2)", [name, age]);

// Vulnerable: user input becomes part of the SQL text
await db.query(
  "INSERT INTO students (name, age) VALUES ('" + name + "', " + age + ")"
);

// Fixed: values are sent as parameters, separate from the SQL
await db.query("INSERT INTO students (name, age) VALUES ($1, $2)", [name, age]);

A review comment might say: "This builds the INSERT from request data. Pass name and age as parameters ($1, $2) so the driver keeps them out of the SQL."

The same check applies to shell commands, NoSQL filters, ORM queries and HTML templates. OWASP's A05:2025 Injection says detection "is best achieved by a combination of source code review along with automated testing."

Secrets in code

Code models can repeat secrets they saw in training. Huang et al. built test prompts from GitHub files that contained credentials and found that code completion tools could return memorized secret strings. Two of the credentials they found were valid.

In review, look for strings that look like keys, tokens or passwords in code, tests, fixtures and config, and for .env files added to the diff. A04:2025 asks it directly: "Are crypto keys checked into source code repositories?" If a real secret was committed, revoke or rotate it first. Deleting the line in a later commit leaves the secret in git history.

Outdated and hallucinated packages

Models also recommend packages that don't exist. Across 2.23 million package references generated by 16 models, Spracklen et al. found that 19.7% named packages that don't exist, including 205,474 unique made-up names. The names repeat: when the researchers re-ran 500 prompts that had produced a hallucination, 43% of the hallucinated packages came back in all 10 runs.

That repeatability is what makes an attack practical. Someone finds a name models keep inventing, publishes a malicious package under it, and waits for an install. The attack is sometimes called slopsquatting, and OWASP's LLM09:2025 Misinformation describes the same scenario.

Outdated suggestions are the quieter version of the problem. Models know less about recent releases, and A03:2025 Software Supply Chain Failures treats software that is "vulnerable, unsupported, or out of date" as a risk in its own right.

In review, check every new dependency. It should exist under that exact name, be the package you meant, be maintained, be pinned in the lockfile, and have no known vulnerabilities in a database such as OSV.

Overly broad permissions

A wider grant is the quickest way to make a permission error go away, so check every grant a change adds. OWASP's A01:2025 Broken Access Control lists "Violation of the principle of least privilege" and CORS settings that let untrusted origins call an API. A02:2025 adds "improperly configured permissions on cloud services."

If the code connects a model to tools, OWASP's LLM06:2025 Excessive Agency describes the agent version of the problem. Its example is an extension that only needs to read data but connects with an identity that can also update, insert and delete.

One finding cuts the other way. In Pearce et al.'s file permission scenarios, Copilot did well, and none of its top suggestions were vulnerable. File modes are only one kind of permission, though, so read the cloud policies, CORS settings, token scopes and database roles in a diff with the same care.

Code nobody fully read

The last risk is about people. Perry et al. found that AI users were more confident in code that was less secure. Fluent code reads as finished code. OWASP's A03:2025 is blunt about the control that prevents this: "No single person should be able to write code and promote it all the way to production without oversight from another human being." An agent that writes a change, plus a person who approves it without reading it, leaves the same gap.

Business logic is where unread code hurts most, because judging it takes knowledge of the domain. OWASP's Secure Code Review Cheat Sheet lists business logic flaws, authorization logic and cryptographic misuse among the areas where human review has the advantage.

A security review checklist for AI-written changes

Use this on any pull request where an agent wrote a meaningful part of the diff. It draws on OWASP's cheat sheet and adds items for the risks above.

Before reading the code

• The pull request says which parts an agent wrote, and the author has read them.

• Static analysis, dependency and secret scans ran on the branch, and their findings are triaged.

• You know which trust boundaries the change touches: new endpoints, new inputs, new outbound calls, new permissions.

Input handling and injection

• Every query uses parameters or a query builder. No SQL, shell command or template is built by string concatenation from input.

• Inputs are validated on the server with allowlists, and output is encoded for its context.

• File paths from input are normalized and kept inside an allowed directory.

Authentication and access control

• Every new route or handler checks who the caller is and whether they may act on this specific record.

• Access is denied by default, and admin functions are protected.

• Session cookies are set HttpOnly, Secure and SameSite.

Secrets and crypto

• No keys, tokens or passwords in code, tests, fixtures or config.

• Passwords are hashed with a slow, salted algorithm, not MD5 or a single SHA-256 round.

• Tokens come from a cryptographically secure random source, and certificate checks stay on.

Dependencies

• Each new package exists, is the one intended, is maintained and is pinned in the lockfile.

• New and bumped versions have no known vulnerabilities.

Permissions and configuration

• New cloud policies, database roles and API tokens grant only what the code uses.

• No wildcard CORS on endpoints that accept credentials. Debug modes and verbose errors are off in production config.

Errors, logging and tests

• Errors fail closed and don't return stack traces to users.

• No secrets or personal data in logs. Failed logins and denied access are logged.

• Tests cover the failure paths: no session, another user's record, malformed input.

Automate the pattern checks, review the rest

Run scanners on every pull request so reviewers start from a known baseline: static analysis (SAST) for risky code patterns, software composition analysis (SCA) for dependencies, and secret scanning for credentials. Static analysis tools include GitHub's CodeQL, Semgrep Code, Snyk Code and SonarQube Server. Dependabot alerts and OSV-Scanner find dependencies with known vulnerabilities. GitHub secret scanning checks a repository's whole Git history for keys, passwords and tokens, and it runs free on public repositories.

OWASP's page on source code analysis tools says static analysis scales well and finds well-known flaws such as SQL injection, but has trouble with authentication problems, access control issues and insecure use of cryptography, and produces high numbers of false positives. Those gaps are where human review time should go.

Where AI code review fits

AI code review reads what the code does, so it can flag security issues such as injection, leaked secrets, missing authorization checks and unsafe deserialization. It takes the first pass on each pull request, so human reviewers can spend their time on design and intent. Installed as a GitHub app, cubic reviews every pull request in minutes and leaves inline comments with suggested fixes. Our guide to AI code review explains how AI review works and how to measure a reviewer.

For specific risks, cubic publishes ready-made review rules. Flag Security Vulnerabilities catches hardcoded secrets, SQL injection, XSS, weak crypto, unsafe deserialization and plain-HTTP calls. Require API Auth, Validation, and Tenant Scoping flags endpoints that skip auth or trust a client-supplied tenant or account ID, and Prevent Sensitive Data in Logs flags logs and error reports that expose passwords, tokens or card data. Each rule checks every pull request, and you can paste it into AGENTS.md so your coding agent follows it while it writes code.

Our codebase scans run agents across a whole repository instead of one pull request. A scan of vinext, Cloudflare's AI-written reimplementation of Next.js, found an auth bypass: named proxy exports in proxy.ts were silently ignored in production builds, so auth checks that worked in development disappeared in production. The project had more than 1,700 Vitest tests and 380 Playwright end-to-end tests, some ported from the Next.js test suite, and the bug still got through. In OpenClaw, an open-source AI agent with much of its code written by AI, a scan found critical, exploitable vulnerabilities, and we built fixes for the four most severe. Most were business logic errors, such as a suffix match that should have been exact.

The AI code security and quality series

Each post in this series goes deeper on one part of the topic:

• OWASP Top 10: the ten risk categories in the 2025 list, what changed from 2021, and what each looks like in code.

• Secure code review: a practical guide and checklist for reviewing code for security, grouped by risk.

• Vibe coding security: the risks in AI code nobody reads line by line, from leaked keys to fake packages, and how to review it before merge.

• Code quality metrics: which metrics research ties to defects found after release, and which ones teams game.

• Cyclomatic complexity: how to count McCabe's metric by hand and how cognitive complexity differs.

• Code smells: twelve common smells, each with an example and the refactoring that fixes it.

• Technical debt: what it is, how teams measure it, and how to pay it down.

FAQ

How secure is AI-generated code?

Less secure than it looks. In one peer-reviewed study, about 40% of the programs GitHub Copilot wrote for high-risk scenarios were vulnerable, and in a user study, people with an AI assistant wrote less secure code while feeling more confident about it. Rates vary by model and by type of weakness, so treat AI-written code as unreviewed until someone has read it.

Can AI-generated code be trusted in security-critical systems?

Only after the same review and testing you would require of a human author. For authentication, payments, cryptography and access control, have a person read every line, require tests for the failure paths, and don't let an agent merge its own changes. OWASP's 2025 Top 10 says no single person should be able to write code and promote it to production without oversight from another human, and the same principle applies to an agent.

What are the biggest security risks of AI-generated code?

Insecure patterns and defaults, injection, hard-coded secrets, outdated or hallucinated dependencies, and overly broad permissions, plus the risk that nobody reads the code closely. Hallucinated packages are the newest of these: in one study, 19.7% of the package references produced by code-generating models named packages that don't exist.

What does a secure AI coding setup look like?

Scanners on every pull request (SAST, SCA and secret scanning), a review step no agent can skip, lockfiles plus a check that each new dependency exists and is the one intended, and least-privilege credentials for both agents and the code they write. Write down a review checklist so reviewers apply it the same way on every AI-written change.