This website uses cookies

Read our Privacy policy and Terms of use for more information.

JBS Weekly

I sat in on a B-Sides Orlando session last week by Aakash Abhay Yadav about securing AI agents, and October is Cybersecurity Awareness Month, so this issue goes there. The talk's core claim is that approving an agent once tells you very little about what it does next. Most of this week's stories turned out to be about the same problem.

This Week’s Signal

Yadav started with a problem I recognize. You review an agent like you would an app, approve it, and a week later it does something you never approved. The risk sits in each individual tool call the agent makes.

The usual fix is a person clicking approve. Anthropic has reported that Claude Code users approve 93% of permission prompts, and the session used that number to show why the click turns into a rubber stamp.

Tuesday's story covers the other half. In a PNAS experiment with nearly 1,000 students, an AI tutor lifted practice scores 48%, then unaided exam scores came in 17% lower than for students who never had it. Anthropic found a similar gap in developers, 50% against 67% on a follow-up quiz. My read: the more you lean on the tool, the worse you get at checking it.

Wednesday's subscription audit held up for a different reason. The limits were set before anything ran (read-only access), and the person signed off on every cancellation.

So skip the extra approval prompts. Set a few hard rules in advance, and save the human for the few decisions that matter.

The Playbook: The Read, Change, Send Check

Yadav's framework for security teams has four stages: normalize, place, decide, and learn. This is a version you can run on one agent in about fifteen minutes, whether it is a chatbot with email access or an automation you built yourself.

Work at the level of actions, not apps. One example from the session was an agent holding an API key broader than intended that deleted production databases in nine seconds. A publicly reported April case fits that pattern: an agent found a token with more access than intended and wiped a company's production database and, as reported, its backups.

1. Sort what it can do into three verbs. Write down everything the agent can Read, Change, and Send out. Every tool, however fancy, boils down to those three.
2. Look for the lethal trifecta. Can it read private data, read content you don't control (emails, web pages, support tickets, shared files), and send something outside? If all three are yes, remove one before it runs. Cutting Send is usually easiest.
3. Write two or three never-rules in plain words. Never send data to an address I haven't approved. Never delete. Never edit its own settings. If your tool can't enforce a rule, that is a finding. It is only a request.
4. Make the risky action reversible. Move instead of delete, draft instead of send, and keep backups somewhere the agent can't reach. Yadav's version rewrote delete commands into a move to a temporary folder.
5. Run it watch-only for a week and read the log. Every call that surprised you becomes a new rule.
6. Pick a stop rule. Pause for yourself only when the agent tries a new kind of Send or touches something sensitive. Keep approvals rare so they still mean something.

From The Podcast

This is your one email for the week's episodes. Tuesday and Wednesday pair most closely with the Signal above.


- AI Lifted Practice Scores 48% and Cut Exam Scores 17%: the hint-first setup that kept the gains and most of the skill.
- Use AI to Audit Your Subscriptions and Cut $1,285 a Year: why read-only access comes first.
- Brooks Running Cut Acquisition Costs 53% With AI Ad Variations: a 3x3 creative test sized for a small ad budget.
- Claude Sonnet 5.5's $2 Price Hides a $7.60 Task Cost: test any new model on your own work before you switch.

Tool Worth Trying

The AI Agent Lockdown Checklist

A free, checkbox-style checklist of 7 things to lock down before an autonomous agent touches your real accounts: hardware isolation, account isolation, key rotation, message allow-listing, read-only permissions, model selection, and an incident response plan. Each section is short enough to act on in a few minutes.

The Read, Change, Send check tells you what one agent can do. The checklist covers the setup around it, including what to do if something goes wrong.

Caveat: It is a PDF, not software. Nothing gets enforced until you make the changes yourself.

Joe’s Take

Sitting in that session, I kept thinking about how often I rubber stamp approvals. I got tired of the prompts and eventually put Claude Code into auto mode.

A model change showed me what can go wrong. I switched the agent that builds my podcast assets to a newer model, and it started making thumbnails in an animated style. My instructions said to use one of three reference images, strictly. The instructions didn't change. The model did.

When I run the Read, Change, Send check on my own setup, Claude Code comes closest to hitting all three. The guardrails I set a while back limit what it can Send. Read is the one I need to lock down. Some of my agents have more access to certain APIs than their day-to-day work needs.

I don't think rules should fully replace human review. Even a workflow that looks safe should get a person looking at it at least once a week.

If you do one thing after reading this, open the API permissions your agents use and drop each one to least privilege. Don't give an agent carte blanche to an API.

Tools I Use

n8n — This issue is about what agents do after you approve them. If you build automations in n8n, the Read, Change, Send check is the fifteen-minute audit to run on any workflow before it touches a real account.

VoiceInk — A local AI dictation tool for Mac that transcribes your voice with near-perfect accuracy and runs entirely on your device, meaning nothing you say ever touches a cloud server.

Blotato — Handles the full content distribution side of your business: drop in a topic and it generates platform-specific posts, or feed it existing content and it repurposes it across formats. TikTok videos become tweets, podcasts become blog posts. Includes a scheduling calendar, visual creation tools for carousels and infographics, and publishes natively to 9 platforms with no per-post fees.

Beehiiv — What you're reading right now is published on Beehiiv. If you're thinking about starting a newsletter or moving off a clunky platform, this is the one I'd recommend. 20% off your first 3 months with my link.

Google Workspace — This week's subscription audit worked because access was set to read-only before anything ran. Google Workspace's Business Standard plan includes that same discipline at the suite level, with Gemini Pro and NotebookLM Plus built in, and a 14-day trial plus 10% off your first year.

Descript — Video and podcast editing that works like a text document. You edit the transcript and the media follows. Cuts filler words, cleans up audio, and handles captions automatically. 50% off your first two months on the Creator Plan.

Final Thoughts

Every story this week came back to one question: who is checking the AI's work? It cannot be a tired person clicking approve, and it cannot be a person whose own skills have faded from leaning on the tool. Put the checking into a few rules you wrote while you were rested, and keep your attention for the decisions that matter. Cybersecurity Awareness Month usually means passwords and phishing. With agents, it also means deciding what the thing is allowed to touch.

PS: If you run the Read, Change, Send check on one of your own agents, reply and tell me the first thing it turned up.

Cheers,
Joe

Reply

Avatar

or to participate