Is it safe to let an agent loose on your systems? An honest checklist

Ten questions to ask any agentic tool before it touches your machine — with honest answers from the one I'm building, gaps included.

"Is it safe to let an AI agent run on our systems?"

I get this question constantly — I'm building rysh, an agentic terminal multiplexer, so I would. But the honest answer to the question as asked is: wrong question. "Safe" isn't a property a vendor can hand you. It's a posture you can verify — or can't.

So here's the checklist I'd use to evaluate any agentic tool, including mine. Ten questions. For each: why it matters, and then rysh's actual answer — including the places where the answer is "partially" or "no." Steal the checklist even if you never install rysh; it's the useful half of this article.

The checklist

1. Can I see every action as it happens — not summarized after?

Why: Post-hoc summaries are written by the same model that made the mistake. Live visibility is the difference between catching a bad edit at diff-time and discovering it in prod.

rysh: Yes — this is the core design. Every tool call renders in the pane as it executes; the visible stream is the data path, not an overlay. (Details.)

2. Which operations require explicit approval, and who decided?

Why: "The agent asks permission" means nothing without knowing for what. All-or-nothing approval schemes decay into reflexive yes-clicking.

rysh: Two-tier gates. File edits/writes are preview-first — the exact diff renders, nothing touches disk until you approve. Commits and non-allowlisted shell commands are pre-approved — shown, then run. Read-only commands (git diff, grep, ls, builds, tests) flow freely to protect your attention for the gates that matter. (Details.)

3. What happens with a command the tool has never seen?

Why: This is the fail-open vs fail-closed question, and it's where marketing claims go to die.

rysh: Fails closed. The bash allowlist prompts on anything unrecognized, plus anything that mutates, redirects, uses command substitution, or chains into unsafe territory. The heuristic's failure mode is an unnecessary question, not an unauthorized execution.

4. What exactly leaves my machine on each model call?

Why: Your prompts contain your code, your logs, your env. If the vendor can't enumerate what's in a request body, nobody can.

rysh: The conversation and tool context go to the model provider (Anthropic — rysh runs on Claude, with your own API key). Before that: env_read output is redacted client-side, and SecretNAT swaps credential-shaped values for reversible look-alike tokens by default. We verified with a logging proxy against the real API: dummy Stripe key pasted in, zero occurrences of the real key on the wire, token sk_live_SNAT000001 in its place. (The wire test.)

5. Where do secrets live, and can the provider reverse them?

Why: "We redact" without "where" is theater. Server-side redaction means the server saw it.

rysh: The SecretNAT token↔value mapping lives in memory on your machine only; you can inspect and reverse it locally (##snat list, ##snat get), the provider never can. Honest bound: it's a translation layer, not a vault — no encryption-at-rest, no rotation; restored values exist in memory at tool-execution time by design. Pair it with a real secrets manager. The claim we defend is exactly: the provider never receives the value.

6. What stops a runaway loop?

Why: Agents get stuck — re-grepping the same file, re-running the same failing command — and every iteration costs money and risk.

rysh: A sliding window tracks the last 20 tool calls (name + params hash); the third identical call is blocked. Plus the human loop: the pane makes repetition visible long before the machine backstop trips.

7. Can I kill it instantly, and what are the cancellation semantics?

Why: "Wait for it to finish" is not an answer during an incident.

rysh: Last-prompt-wins: a new prompt cancels any in-flight run at the execution layer. Cancellation is a first-class semantic, not a UI hope.

8. Where is state stored, and can I actually delete it?

Why: Conversation history is a shadow copy of your codebase and your thinking. If you can't locate it, you can't govern it.

rysh: Session state persists locally (embedded NATS JetStream, per-session directories under your own state dir). rysh delete-session removes the session record and its data directory. It's your disk; it's your data.

9. Whose infrastructure does it run on?

Why: Every hop you don't control is a hop you're trusting on faith.

rysh: Self-hostable — the CLI runs on your machine or server with your own Anthropic key; team collaboration features run through a server you can also host yourself. Honest bound: self-hostable is not open source; the code isn't public. Model calls go to Anthropic — that hop exists, minimized by redaction and SecretNAT above.

10. Will the vendor tell me what it doesn't protect against?

Why: This is the meta-question. A vendor with no "what this is NOT" section hasn't thought about their threat model — or has, and won't say.

rysh: This article is the answer, but to be explicit about known limits: SecretNAT's detected tier is heuristic (register secrets you care about); a shared pane is a screen share — govern it like one; approval gates can be waved through by a user who stops reading; and no layer here makes the model correct — it makes mistakes visible and cheap to catch.

Scoring it

No tool — mine included — gets a perfect ten without asterisks. What you're actually scoring is whether the vendor's answers are specific, bounded, and verifiable. "Enterprise-grade security" is none of those. "Grep the wire and count occurrences of your key" is all three.

The pattern I'd push you toward: prefer tools whose safety story is built from mechanisms you can observe (visible calls, previewed diffs, fail-closed gates, local state) over tools whose story is built from assurances you must trust. The first kind survives your security review. The second kind survives until the incident.

rysh is in private beta — self-hostable, your infra, your Anthropic key, built on Claude. The design-partner program is open: free access, a direct line to the founder, and honestly — teams who ask hard versions of these ten questions are exactly who we want stress-testing the answers. Apply here.

Related: Every tool call visible · SecretNAT: the wire test