Run one real job through an installed AI agent skill, then immediately decide: keep it, fork it, or delete it. That's the one-job test, and it exists because bloated skill libraries quietly dilute agent focus. Tools like Claude Code and Codex only surface so much of a skill library at once, so a crowded library doesn't just waste tokens, it degrades performance on the tasks you actually care about.
Less is measurably more
A WorkOS talk gave this the hardest numbers in the research window. Nick Nisi generated 10,000+ lines of skills straight from WorkOS documentation, watched task accuracy sit at 77%, then cut it down to 553 hand-written "gotchas" specific to the product. Accuracy jumped to 97%, and eval runtime dropped from 68 minutes to 6. The pattern isn't unique to that one case: agents already know how to write code in general, they just need to know your specific landmines. A comprehensive, multi-thousand-line skill library doesn't teach the model more. It sends it on wild goose chases.
Boris Cherny, who created Claude Code, put the underlying principle bluntly at YC Startup School: "every 6 months, delete your claude.md file, delete your skills, and delete your hooks. Then see what the model does. It might surprise you." His own team stripped 80% of Claude Code's system prompt for the Opus 5 release, and the model got smarter, not dumber. Scaffolding built for a weaker model becomes dead weight once the model improves, and nobody goes back to remove it.
What a skill actually is
Skills are the procedural memory layer of an agent system. MCP handles external tool access, RAG provides semantic facts, and skills dictate how to perform a task, in what order, with what judgment. They're standardized as markdown files with YAML front matter, and they load via progressive disclosure: metadata first, then detailed instructions, then optional scripts or references, only pulled in when the model's reasoning matches a skill to the task at hand. A Firecrawl roundup of the best Claude Code skills names the same discipline directly: one clear job per skill rather than a skill trying to cover multiple workflows, paired with concrete worked examples over abstract rule lists.
From instructions to enforcement
Agents will occasionally fail, skip steps, or fabricate progress if given too much latitude, and the response showing up across builders is to enforce boundaries in code rather than just instructing via prompt. One example: a TypeScript-based state machine that demands hard evidence before letting a task advance, cryptographically hashing test outputs to confirm a test suite actually ran, or recording video of a UI fix rather than trusting a text description of it.
Not everyone is deleting
The pushback is worth including. One response to Cherny's advice: "Why would too much info be bad? Also, my business evolves so I'm updating them every 2-3 months anyway. I feel like that would be commonplace... or maybe it's just my Wild West industry." That's a fair point specifically for domains that change fast. The delete-everything advice assumes a relatively stable domain where old scaffolding genuinely becomes obsolete, which isn't every use case.
The playbook
Keep a claude.md under 200 lines defining core context, project rules, and folder structure, read automatically at the start of every session. Add a rule enforcing planning mode: present a written plan and wait for approval before starting any multi-step task. Build skill content around specific gotchas and landmines rather than general instruction the model already knows. For error analysis at scale, use automated evaluation pipelines to cluster failures by mode instead of reading failed logs one at a time. And run the one-job test on every skill you install: if it doesn't earn its place on a real task, fork it down to what actually works or delete it outright.
FAQ
What is the "one-job test" for AI agent skills?
Run a single real-world task through an installed skill, then immediately decide to keep it, fork it into something narrower, or delete it, rather than letting skill libraries accumulate untested.
Why would deleting skills improve an agent's performance?
Crowded skill libraries dilute what an agent can surface at once. One documented case cut a 10,000-line skill library to 553 targeted lines and saw task accuracy improve from 77% to 97%, with eval runtime dropping from 68 minutes to 6.
How often should I review my claude.md and skills?
Claude Code creator Boris Cherny recommends every 6 months: delete claude.md, skills, and hooks, then see what the model does without them, since scaffolding built for an older, weaker model version often becomes unnecessary dead weight.
What format do AI agent skills use?
Markdown files with YAML front matter, loaded via progressive disclosure (metadata first, then detailed instructions, then optional scripts) so the agent only reads a skill's full content when it's actually matched to the task.
This piece is the companion writeup to the Daily AI Pulse episode "AI Agent Skills: The One-Job Test for Keep, Fork, or Delete." Watch it on YouTube, or dig into the source links above for the full picture.
More AI breakdowns for solo builders and small teams 👉 joebuildsai.com