Judge-decided reset and resume: letting an LLM decide when the loop is done

Exit conditions for agent loops are brittle. rysh adds an LLM judge that rules done, resume, or reset — inside hard budgets.

Every agent loop ends one of three ways: it finishes, it runs out of budget, or you kill it in disgust.

The interesting engineering question is the first one. Who decides finished?

For "make the tests pass," the answer is easy — an exit code decides. For "build me a shortlist of twenty podcast guests who actually fit this show," there is no exit code. There is no regex. "Done" is a judgment call, and most agent frameworks quietly hand that judgment to the worst possible party: the agent doing the work.

The brittle ways to end a loop

You've probably tried all of these:

• Fixed iteration counts. max_steps: 25. Why 25? Nobody knows. Too low and you truncate real work; too high and you burn tokens polishing a finished answer.
• String matching on output. The agent says "TASK COMPLETE" and you grep for it. Congratulations, you have taught a language model that saying the magic words ends the shift.
• Vibes. You sit and watch it. Which works — and also means you don't have an autonomous agent, you have a very expensive screensaver with a supervisor.

The failure mode is the same each time: the loop's exit condition and the loop's goal are different things, and the gap between them is where agents either quit early or run forever.

The shape rysh uses: do/while with a judge

In rysh, a governed loop has two legs.

The work leg runs the actual task — an autonomous agent spawned from a markdown skill file, or a browser recipe under ##auto web — inside hard budgets you can say out loud (per-pass, per-run, per-campaign caps; the pages, books, and shelves framing).

Then the judge leg. When the loop pauses — because the work claims it's done, or because a budget ran dry — a separate LLM pass reads the goal and the actual artifacts (the shortlist file, the diff, the test output) and returns one of three verdicts:

• done — goal met, stop, ship the result to whatever comes next (notify, on_success, a human).
• resume — goal not met, but the partial output is good. Continue from it.
• reset — goal not met, and the partial output is part of the problem. Keep the goal, throw the context, start clean.

Resume is the underrated one

Reset is obvious. Resume is where the real savings live.

Here's a concrete case from our own use: a browser agent scouting podcast guests gets interrupted mid-run — network hiccup, budget pause, laptop lid. The naive restart begins from zero: re-search, re-open, re-evaluate the same profiles, and probably produce duplicate rows.

##auto web resume does the grounded thing instead: the next run starts with the partial shortlist as input. The agent's first job is to read what already exists, then extend it — no re-visiting profiles it already judged, no duplicates, no amnesia. The partial artifact isn't debris to sweep up; it's the checkpoint.

Reset is for the other case — when the judge concludes the direction was wrong. Fifteen candidates who all fail the same fit criterion isn't a checkpoint, it's a rut. Grounding the next run on it would reproduce the rut. So: same goal, fresh context.

An LLM judge is decent at exactly this distinction, because it's a reading-comprehension question about evidence — not a self-assessment.

Why the worker doesn't grade its own homework

Because you already know how that ends. A worker agent asked "are you done?" is an optimist with an incentive: saying yes ends the task. "Task complete ✅" above a task that is not complete is the most familiar screenshot in this industry.

The judge is a separate pass with a separate framing. It doesn't see the worker's self-narrative ("I have successfully completed…"). It sees the goal and the artifacts, and it answers a narrower question: does this evidence satisfy that goal? Same model family, different seat. The separation is the point.

Honesty: the judge is a signal, not proof

Let's be precise about what you're getting. The judge is an LLM. It can pass work that should fail and fail work that should pass. If your pipeline treats its verdict as ground truth, you've just moved the gullibility one level up.

That's why the judge never runs alone in rysh:

• Hard budgets bound the damage. A wrong "resume" costs at most the remaining budget, not your weekend.
• takeover_when: 90 pulls a human in when 90% of the budget is spent and the goal still isn't met — before the loop dies quietly.
• Everything happens in a pane. The work, the verdict, the reset — all visible tool calls in a terminal you can watch, share, or scroll back through. The judge decides the cheap question (continue or stop). Humans keep the expensive one (ship or don't).

And the standing disclosure: rysh runs on Claude with your own API key. The judge is a Claude call you control and can read — self-hostable, on your infra. It's not a proprietary oracle; it's a governed second opinion.

Composing it

Because the verdict is just a decision point in a loop, it composes with everything else in the workspace: ##cron to schedule the loop, notify to tell you how it ended, on_success to chain the next step, and the pane's history if you want to audit why the judge ruled the way it did.

A loop with a goal, a budget, a judge, and a takeover threshold isn't an experiment anymore. It's a process you can hand real work to.

We're recruiting design partners. If your team is running agent loops and wants them governed instead of hopeful, come build this with us: rysh.ai/design-partner

Related: Pages, books, and shelves: budget your agents like a library · No goal, no loop