AI Coding Agent vs a Flaky Dev Server

The worst half hour you can have with an AI coding agent dev server is the one where the server died twenty minutes ago and neither of you noticed. The agent requests a page, gets a connection refused or a cached build from before its last three edits, concludes the code is wrong, and starts fixing code that was never broken. By the time you look, the diff is large, confident and pointed entirely in the wrong direction.

This is not a model failure. It is a missing precondition, and the fix is boring and permanent.

Why an AI coding agent dev server problem gets blamed on your code

An agent reasons from observations. Feed it a stale observation and the reasoning stays sound while the conclusion goes wrong. It has no independent way to know that the page it just loaded was built before its edit landed, so a correct change that appears to have no effect reads, reasonably, as a change that did not work.

What follows is the expensive part. Having decided the change was ineffective, the agent escalates: it tries a bigger version of the same change, adds a workaround, then starts touching adjacent files. This is how a one-line edit becomes a twelve-file diff, and it is a close cousin of getting stuck in a loop, with the difference that here the environment, not the model, is supplying the bad feedback.

Four symptoms that mean environment, not code

• The change has no visible effect at all. Not the wrong effect, none. Real logic bugs usually change behaviour, just not the way you wanted.

• The agent's next three attempts are escalations of the first. Bigger hammer, same nail. Genuine debugging tends to narrow rather than widen.

• Errors mention the transport, not your logic. Connection refused, socket hang up, 502, empty response. Those are server facts.

• A hard reload or restart changes the story. If restarting the dev server makes the original fix work, nothing after the first edit was real.

Any two of those together, stop the agent. Do not let it keep diffing while you investigate, because every additional edit makes the eventual revert harder to scope.

The health-check-first rule

The permanent fix is to make verification a precondition of testing rather than an afterthought. Instead of letting the agent poke the app directly, give it one script and tell it that the script is the only legitimate way to check behaviour.

#!/usr/bin/env bash
# check.sh: prove the server is current, then test.
set -euo pipefail

PORT=${PORT:-5173}
URL="http://localhost:$PORT"

# 1. Is anything listening?
if ! curl -fsS --max-time 3 "$URL/__health" >/dev/null 2>&1; then
  echo "DEV SERVER NOT RESPONDING on $PORT. Start it, do not edit code." >&2
  exit 70
fi

# 2. Is it serving the current build? Compare a build stamp written at startup.
served=$(curl -fsS --max-time 3 "$URL/__health" | tr -d '\n')
latest=$(git rev-parse --short HEAD 2>/dev/null || echo nogit)
if [ "$served" != "$latest" ]; then
  echo "STALE BUILD: server has $served, repo is at $latest. Restart it." >&2
  exit 71
fi

# 3. Only now do the real work.
npm test --silent

Exit code 70 means the environment is down and 71 means it is stale. Both are distinct from a failing test, which is the entire point: the agent can be instructed to treat a non-zero exit in the seventies as "fix the environment or stop and ask", never as "change the code".

Then say so plainly in the instructions:

Run ./check.sh to verify behaviour. Never curl the app directly.

If check.sh exits 70 or 71, the dev server is down or stale. Restart it and
run check.sh again. Do not edit application code in response to a 70 or 71.
If it still fails after one restart, stop and tell me.

Making the server itself less flaky

The health check contains the damage. Reducing the frequency is a separate job and mostly comes down to three things.

First, watcher limits. On large repositories a file watcher silently stops noticing changes once it runs out of handles, which produces exactly the stale-build symptom; raising the inotify limit on Linux or switching to polling fixes it, and most bundlers document the trade-off, including Vite's troubleshooting guide.

Second, hot reload on files that cannot hot reload. Changes to configuration, environment variables, service workers and server entry points usually need a full restart, and a partial reload leaves you running half the old process. If your stack has a known list, put it in the instruction file so the agent restarts instead of reloading.

Third, port reuse. An orphaned process from a previous session holding the port means the new server binds somewhere else or fails quietly while the agent happily tests the old one. A pre-flight that kills stale listeners removes a surprising share of these:

# dev.sh: never inherit a previous session's server.
PORT=${PORT:-5173}
pid=$(lsof -ti ":$PORT" || true)
if [ -n "$pid" ]; then
  echo "killing stale listener $pid on $PORT"
  kill "$pid" 2>/dev/null || true
  sleep 1
fi
exec npm run dev -- --port "$PORT" --strictPort

The --strictPort flag, or whatever your bundler calls it, is the quiet hero. Without it a blocked port means the server helpfully starts on the next one up, and now the agent is testing an address nothing is serving while a perfectly healthy server runs next door.

What this looks like when it goes wrong

A concrete shape, because recognising it later is most of the value. You ask for a change to a form validation rule. The agent edits the validator, loads the page, and sees the old error message. It widens the change to the form component. Still the old message. It adds a key to force a remount. Still the old message. It starts editing the state store. Twenty minutes and nine files later the message has not changed, because the server stopped watching files after the first edit.

Every individual step there is defensible. The compounding is what hurts, and notice that the agent's own confidence grew as the evidence accumulated: each failed attempt made a deeper cause seem more likely. That is why the environment check has to be a precondition rather than something you remember to do when things look odd. By the time things look odd, the diff is already expensive.

If flakiness persists, a reproducible environment is the heavier answer: a dev container for your coding agent trades a little setup for a server that starts the same way every time. The broader question of where the agent runs at all is covered in cloud versus local agents, and the tool landscape in picking an AI coding tool.

Frequently asked questions

How do I know whether the agent already fixed the wrong thing?

Restart the server, revert the agent's diff, re-apply only its first edit, and test. If that passes, everything after the first edit was a response to bad feedback and can go.

Is a build stamp endpoint worth adding to a real app?

Yes, and not only for this. An endpoint returning the current commit is useful in staging and production for exactly the same reason: it answers "what is actually deployed" without guesswork. Keep it out of public routing if the commit hash is sensitive.

Should the agent be allowed to restart the server itself?

Generally yes, with a limit of one restart per check, which stops a restart loop. What it should not be allowed to do is edit code in response to an environment failure.

Does this apply to tests in continuous integration too?

The staleness problem mostly disappears there, since each run starts clean. The transport-error problem does not: a flaky service dependency in a pipeline produces the same misattribution if the agent is reading the logs.