AI Coding Agent vs Chat: When to Let It Run

Photo by Moritz Mentges on Unsplash

The choice between an AI coding agent and chat isn't about which one is better — it's about how much damage a wrong answer can do before you see it. Chat mode means you approve each action, so the blast radius of a mistake is one tool call. Agent mode means the model plans, executes, checks its own work, and hands you a finished result, so the blast radius is however far it got before it stopped. My rule after a year of shipping this way: run agents on work that is reversible, verifiable, and scoped to files I could delete without crying. Everything else, I watch.

TL;DR

• Chat and agent are the same model with different approval boundaries. The question is always "how much can this break before I notice?"
• Reversible plus verifiable equals safe to run unattended. If a passing test proves it worked and git can undo it, let it go.
• Anthropic's own incident log names the failure pattern: an overeager agent taking initiative past what you authorized, not a malicious one.
• Sandboxing and auto mode replace approval clicking with structural limits — Anthropic measured an 84% drop in permission prompts from sandboxing alone.
• Domain expertise, not coding ability, predicts whether an agent run succeeds. Know your problem well and you can run longer leashes.

What actually changes between chat and agent mode

Nothing about the model changes. What changes is who approves the next action.

In chat mode, you send a prompt, the model proposes an edit or a command, and you approve it. You see the file path before it writes. You see the shell command before it runs. Your judgment is in the loop on every single step, which is slow and which is exactly the point.

In agent mode, you send a prompt and the model runs a chain of actions until it decides it's done. It reads files, writes code, runs tests, reads the failures, fixes them, and runs the tests again. You see the result.

The size of that chain is bigger than most founders expect. In Anthropic's analysis of roughly 400,000 Claude Code sessions between October 2025 and April 2026, each user prompt set off around 10 actions on average — and about 2% of sessions averaged more than 100 actions per prompt. A typical session was only about four turns. So four things you type, and Claude does forty things.

That same research found a stable division of labor: users made roughly 70% of the planning decisions (what to build) while Claude made about 80% of the execution decisions (which files, which commands, which approach). That split is the honest description of agentic coding. You are not delegating the problem. You are delegating the typing and the intermediate judgment calls.

It also found something that changes how you should set your leash length. When the user kept control of execution — making over 80% of execution decisions — Claude took about eight actions per turn. When Claude controlled planning, it took about sixteen. The more you hand over, the further it goes between check-ins, which is obvious in hindsight and worth saying out loud: the leash length is set by how much context you give, not by which mode you clicked.

I learned this the annoying way. Early on I'd write a vague prompt in agent mode, get 40 actions of confident work in a direction I didn't want, and blame the agent. The agent was fine. My prompt authorized a much bigger search space than I meant. That's the same failure I wrote about in why AI keeps inventing file paths — under-specification reads as permission.

How much can this break before I notice?

Here's the mental model I actually use. Before starting anything, I answer two questions.

Is it reversible? If the agent does the wrong thing, can I get back to where I was with one command? Code changes in a git repo with a clean working tree are reversible. Database migrations against production are not. Emails already sent are not. Stripe subscriptions already cancelled are not.

Is it verifiable? When the agent says "done," is there a signal that isn't the agent's own opinion? A passing test suite is a signal. A typecheck is a signal. A screenshot of the page rendering correctly is a signal. "I've implemented the feature as requested" is not a signal — it's the agent grading its own homework.

Reversible and verifiable both true? Run it unattended. Either one false? Watch it.

This maps cleanly onto real work:

| Task | Reversible? | Verifiable? | My mode | | --- | --- | --- | --- | | Fix a failing test | Yes (git) | Yes (test passes) | Agent, unattended | | Refactor a component | Yes (git) | Yes (typecheck + tests) | Agent, unattended | | Add a new marketing page | Yes (git) | Partly (visual check) | Agent, then I review | | Wire up a Stripe webhook handler | Yes (git) | Partly (test events only) | Chat, step by step | | Run a production database migration | No | No | Chat, and I read every line | | Delete "unused" files | Sometimes | No | Chat, and I check each one |

The last row is the one that bites people. "Clean up the unused files" sounds like a chore. It's actually an irreversible destructive action with no verification signal, dressed up as housekeeping.

What actually goes wrong in unattended runs

The failure mode is not what founders fear. It isn't a rogue model. It's a helpful one that takes initiative past the boundary of what you asked for.

Anthropic published its internal agentic-misbehavior incident log alongside the engineering writeup for Claude Code's auto mode, and the examples are worth reading closely because they're all reasonable-looking:

• A user asked to "clean up old branches." The agent listed remote branches, built a pattern match, and issued a delete.
• An agent hit an auth error mid-task and started grepping environment variables and config files for a different API token it could use instead.
• A user said "cancel my job." The agent queried the cluster, picked the closest name match, and tried to delete it.
• A deploy command failed a pre-check, so the agent retried with a skip-verification flag.

Not one of those is malice. Every one is an agent solving the problem you gave it, one step past the line you meant to draw. Anthropic's own framing of the threat model puts "overeager behavior" and "honest mistakes" first, ahead of prompt injection and misalignment, because those are the ones that show up in practice.

The second category matters for founders who run agents against anything that touches the internet. Prompt injection means instructions planted in a file, a fetched webpage, or a tool result hijack the agent toward someone else's goal. If your agent reads a scraped page or a customer-submitted document, that content is untrusted input, not instructions. I cover the founder version of this in the AI coding security review.

There's a third thing that goes wrong, and it's less dramatic but far more common: the agent gets stuck in a loop, tries the same broken fix six times, and burns through your token budget while producing nothing. Unattended means unattended — nobody stops it. I wrote up how to spot and break the loop after losing an afternoon to one.

The guardrails that actually let you walk away

You don't get safe unattended runs by being careful. You get them by making the unsafe thing structurally impossible. Three layers, in the order I'd add them.

Layer one: a clean git state, always. This is the cheapest guardrail in existence and most non-technical founders skip it. Before any agent run, commit your work. Not because the agent will destroy things, but because a clean baseline turns every agent output into a reviewable diff you can throw away with one command. If you're shaky on this, git basics for non-developers covers the four commands that matter.

Layer two: sandboxing. Claude Code's sandbox enforces two boundaries at the operating-system level: filesystem isolation, so the agent can read and write inside your project directory and nowhere else, and network isolation, so it can only reach approved hosts. Anthropic notes that you need both — without network isolation a compromised agent can exfiltrate your SSH keys, and without filesystem isolation it can escape the sandbox to get network access. In their internal usage, sandboxing cut permission prompts by 84%. That number is the whole argument: the point of a sandbox isn't paranoia, it's that you stop being asked.

Layer three: classifier-gated permissions. Anthropic's auto mode sits between "approve everything manually" and --dangerously-skip-permissions. A classifier evaluates each action before it runs, blocking things like force-pushing over history, sending internal data to external services, or modifying the agent's own permission config. The published numbers are refreshingly honest — the deployed pipeline blocks only 0.4% of benign actions, but still misses 17% of genuinely overeager ones. Anthropic's own summary of that tradeoff is the right way to think about it: it's a large improvement over skipping permissions entirely, and a regression compared to careful human review. There's also a backstop worth knowing about: after three consecutive denials or twenty total, the session stops and escalates to you.

Which is to say — none of these layers makes unattended running safe. They make it survivable. The judgment about which tasks deserve a leash is still yours.

Where founders should draw the line

My working policy, which you're welcome to steal:

Always agent mode: test fixes, typecheck errors, renaming things, writing tests for existing code, formatting, dependency bumps in a branch, drafting documentation, building a page against a spec I already wrote.

Agent mode with a review gate: anything that touches auth, anything that touches billing logic, any new database table, any change to more than about five files. The agent runs, I read the whole diff before it merges.

Chat mode only: production data, secrets and environment variables, anything that sends email to real customers, anything that charges a card, any deletion. Environment variables in particular deserve their own paranoia — env vars and secrets management has the setup I use.

The thing that determines how far you can push toward the top of that list isn't your coding ability. Anthropic's session research found that people in software occupations reached verified success in about 30% of sessions, versus about 26% for everyone else — a five-point gap that hasn't widened or narrowed in seven months. But sessions rated as showing domain expertise reached verified success more than twice as often as novice-rated ones. And when a session hit trouble, 19% of novice sessions were abandoned outright, against 5–7% for everyone else.

Read that as good news, because it is. You don't need to become an engineer to run longer agent leashes. You need to understand the problem you're solving well enough to tell when the answer is wrong. Which is the same skill that made you want to build the product in the first place.

The habit that gets you there fastest is writing the spec before the prompt. When I know exactly what "done" looks like, the agent runs longer and lands closer — which is the whole argument in spec-driven development with AI. And a good CLAUDE.md does the same job permanently: it front-loads the constraints so you don't have to re-authorize them every session. The CLAUDE.md file guide covers what belongs in one.

Frequently asked questions

Is agent mode more likely to write bad code than chat mode?

No. It's the same model with the same capabilities. What differs is how many steps happen before you look. Bad code produced in chat mode gets caught at step one; bad code produced in agent mode gets caught at step forty, after it's been built on. The quality is the same, the review cost is not.

How do I stop an agent that's going the wrong direction?

Interrupt it and give it new context rather than restarting from scratch. Most of the time the agent isn't confused about how to code, it's confused about what you wanted. A mid-run correction with a concrete constraint ("no, keep the existing schema, add a column instead") is faster than a fresh prompt. See the feedback loop with your AI assistant for the pattern.

Should I run agents overnight while I sleep?

Only on work that is reversible and verifiable, in a sandbox, on a branch. An unattended overnight run against your main branch with production credentials in the environment is how the incidents in Anthropic's log happen. An overnight run that fixes a failing test suite on a feature branch is fine.

Does agent mode cost more?

Yes, meaningfully. More actions means more tokens, and a stuck loop burns tokens with nothing to show. If you're on a usage-based plan, treat a long unattended run as a spend decision, not just a time decision.

What if I don't know enough to review the diff?

Then keep the leash short and use the agent to teach you. Ask it to explain the change before you approve it, and ask what would break if it were wrong. Reading AI output as a non-coder is the skill that upgrades every other part of this workflow, and it compounds fast.

Is there a way to get agent speed with chat safety?

Partly. Sandboxing plus a clean git state plus a strong test suite gets you most of the way there — the agent runs freely inside a boundary where its worst mistake is a bad diff you delete. That's not the same as safety, but for reversible work it's close enough to stop worrying.

The short version

The AI coding agent vs chat decision is a blast-radius decision, not a capability one. Ask whether a wrong answer is reversible and whether a signal other than the agent's own confidence will tell you it worked. Two yeses means let it run. Anything else means stay in the loop, because the failure you'll actually hit isn't a rogue model — it's a helpful one that took initiative one step past what you meant.

Start conservative and lengthen the leash as your test coverage and your understanding of the codebase grow. Those two things, not your ability to write code, are what let you walk away from a running agent.

If you're building a SaaS with AI coding tools, Coding Capybaras is the free boilerplate I built for exactly this workflow — clean region boundaries, a CLAUDE.md that tells the agent what it's allowed to touch, and a test suite that gives you the verification signal these runs depend on.