There's an old saying in security: you can’t protect what you don't know exists. Apply that to AI-assisted development, and things get uncomfortable really fast.

We're pasting error messages into LLMs, copying the "fix" back into our IDEs, and hitting merge. But what are we actually merging?

When you paste that AI-generated snippet, you might have just introduced a dependency nobody vetted, a deserialization pattern straight out of a vulnerable 2017 blog post, or an "optimized" SQL query that silently drops indexes on your biggest table. Maybe the model just guessed an environment variable name wrong and hardcoded a real secret instead.

I’ve reviewed PRs where the diff is painfully obvious AI output. There are imports nobody on the team uses and comments written in a voice nobody speaks in. I'll ask the developer, "Hey, why did you use asyncio.gather here?" And they stare blankly: "The AI said it would be faster."

That’s not a velocity win. That’s an attack surface expansion.

The Broken Assumption of Code Review

Traditional code review relied on a soft rule: if I ask you why a line of code exists, you should be able to answer me without checking Google. That assumption is breaking down.

Reviewers are skimming more because PR volume is through the roof. Meanwhile, attackers don't even need to breach your repo if your CI/CD pipeline is happy to merge their payload disguised as a helpful bugfix. Prompt injection in a Jira ticket description leads to a model suggesting a malicious config, which a tired dev copies and pastes. It sounds exotic until you’ve seen the exact ticket.

In the short term, this causes weird prod bugs, performance cliffs, and secrets leaking into logs. In the long term, it hollows out your team's ability to handle incidents. When the 3 AM pager goes off, nobody will know which third of the codebase is alien territory.

Quality and Security are the Same Bet Now

We used to separate these on the org chart. QA sits here, Security sits there, Devs in the middle shipping features. AI completely collapsed that wall. The same rushed merge that skips readability is the exact same merge that skips threat modeling.

Here are the questions nobody is asking on AI-heavy teams:

  • Who is accountable if this generated block mishandles user input?
  • Did we actually run SAST on code the author didn't even mentally parse?
  • Are we logging PII because the model "helpfully" added a few debug print statements?
  • Does our dependency scanner catch packages the model completely hallucinated into requirements.txt?

"Works in the demo" is not a security posture.

What Actually Helps (Practical, Not Keynote)

Forget the polished conference talks. Here’s what actually moves the needle.

Make authors attest to their code.Add a short section to the PR template: "I understand lines X–Y. Block Z is generated, but I verified its behavior against [test/scenario]." It forces a moment of honesty before the merge button turns green.

Run SAST on every PR, no exceptions.Models love unsafe SQL, eval(), and path traversal shortcuts. Let the robot fight the robot.

Separate exploration from merge.Copilot/Cursor or any other AI IDE commits on a spike branch are totally fine. Accidentally merging that unreviewed spike to main is how junk becomes permanent architecture.

Update your security training.Not the OWASP slides from 2019 with a new copyright date. Real scenarios: prompt injection via support tools, leaked API keys in prompts, and supply chain attacks via suggested packages.

Red-team the workflow, not just the app.Social engineering didn't go away; it just moved upstream into the IDE.

The world moved fast on AI code generation. Adversaries are moving just as fast on our misgeneration.

You don’t need to ban AI in your codebase. But you need to stop pretending the green "Build Passed" button is a quality gate—or a security gate. It’s just a light that says the checks you remembered to automate passed.

Everything else is still on you.