How I work with agents: real security vs security theater

I said I’d write about how I work with agents, including the difference between real security and security theater. This is that note. It is not a second retelling of the OpenAI / Hugging Face eval mess. If you want the incident, start there. This one is about my bench.

I use agents constantly. Named ones, with jobs. Not because I’m cosplaying a company, and not because I think a clever roster name is a control plane. Because when everything is “hey AI, do the thing,” I stop knowing who is allowed to touch what.

What “working with agents” means here

On a normal day I am not handing a model the keys to the open internet and a scoreboard. I’m pairing with a small crew that has lanes:

• Someone who keeps the phases honest (discover before dig, quality before “ship it”).
• Someone who builds.
• Someone who writes down what we actually decided.
• Someone who hunts defects and edge cases.
• Someone for the hard edges: sync, auth, local databases, the stuff that bites if you hand-wave it.

The names are a mnemonic. The point is routing. Research does not silently become a production commit. A UI chore does not get to invent a sync protocol. A competitive scan does not get to invent a product launch.

That is already a security posture, even when the work is “draft a webpage” or “move a folder out of limbo.” Blast radius is a habit, not a feature flag.

Real security (the list I actually use)

None of this is glamorous. That is why it works.

1. Humans still own the dangerous verbs.
Agents can draft, search, propose, and even prepare commits. I still decide what gets pushed, what gets published, and what gets a new private repo. “Ask before push” is not politeness. It is egress control for my own reputation and my own secrets.

2. Confirm before mutate.
Builds, dependency bumps, and anything that rewrites the world outside the editor need an explicit yes. Digging in feels productive. Digging in without a gate is how “I’ll pin versions later” becomes the permanent architecture.

3. Hygiene before features.
When a product graduates from sticky notes to a real home, the next wave is not “flutter create and vibes.” It is Dependency Hygiene: toolchain pins, intentional dependencies, written exit criteria. I have a pre-build gate doc sitting in a repo right now specifically because I know my own failure mode. Future-me will try to skip it. Present-me left a door she has to walk through, and informed the agents she is required to actually open that door with her own hand.

4. Secrets get a backup path, not a shrug.
If something sensitive has to leave a file for git or a sanitizing pass, the rule is copy to a local backup that will not sync, then scrub. Never “delete the only copy and hope I remember the console.” That is not drama. That is how you avoid turning a cleanup into an outage.

5. Separate parking lots from product homes.
Ideation can live in a private research hub. Products that have earned a thesis get their own space. Theater oy lists do not ride along inside the email app’s git history. Containment is partly about models. It is also about where the files live and who can see them.

6. Sources over soundtrack.
When I write about AI incidents, primary disclosures and solid reporting beat vibes. If two lab stories look similar from a headline, I check whether they are actually the same failure mode before I mash them into one morality play. Conflation is a kind of insecurity theater: it feels decisive and teaches the wrong lesson.

7. Agents propose; I still read the diff.
If I cannot explain what changed, I do not ship it. Named agents do not get me out of review. They make review possible because the work arrived in a shape I can inspect. They can write code a zillion times faster than I can. That doesn’t absolve me from understanding what is being produced. If I’m not sure what they did or why, I ask, then make sure I’m in clear agreement before I (yes, *I*) say it can ship.

Security theater (what I refuse to confuse with the list above)

Theater looks like:

• A fancy agent org chart with no phase gates and no “who may push.”
• “The model is aligned, so the sandbox is fine,” while the sandbox still has a package door and nobody is watching egress.
• Checklist culture: policies nobody can operationalize, or operational habits nobody wrote down.
• Letting an agent “just fix production” because it is faster than understanding the change.
• Pretending a chat sidebar and an autonomous tool-using agent are the same risk class. They are not. I said that in the incident note for a reason.

Theater is when the language of safety shows up and the controls do not. Real security is often quieter: fewer verbs available without a human, smaller blast radius, slower when slow is correct.

What the July mess changed for me (and what it didn’t)

It did not make me quit agents. I build with them. I write with them. I will keep doing that. I even enjoy doing that.

It did sharpen the sentence I already believed: if you give a goal-directed system tools and reduce the brakes, the adults in the room are isolation, logging, kill switches, and “who gets paged when this traffic is us.” That sentence is about frontier evals. The small version on my bench is: do not give an agent unsupervised push, unsupervised secret access, or unsupervised “upgrade everything” just because the afternoon is going well.

I am not OpenAI’s eval network. I am also not exempt from the pattern. Incentives plus doors plus late detection is how ordinary projects get ordinary incidents.

The practical agreement I want with myself

When I work with agents:

• Route work on purpose.
• Keep humans on publish, push, secrets, and dependency bumps.
• Write the gates down so “later” has to argue with a checkbox.
• Prefer boring containment over dramatic confidence.
• Stay curious about industry failures without turning every headline into either apocalypse or .

Instruments, not vibes. That line is about products. It is also about process.

More soon on the industry side (evals, shipping, the mood). This one was the mirror.

~ Trish