Skill Distill Skill: Reuse a Proven Coding Session

Share

On 21 November 2026, Sybil Solutions published Skill Distill, a Claude Code and Codex skill that turns one successful coding session into a tested procedure for a named open-weight model and agent harness. Its central safeguard is simple: each version faces three clean-room student trials before the user accepts it.

At a glance

What does Skill Distill Skill do?

Skill Distill is not a model-training package and not a prompt generator for a blank task. It takes a coding session that already produced a good result, then asks a teacher agent to write instructions that let a specified student recreate that result from the same prompt and starting state. The project’s own procedure describes this as a loop: write the skill, run the student, inspect the difference, repair the first mistake, and repeat.

That makes its output narrower than a general-purpose “coding best practices” skill. A finished artifact is for one task, one model and one harness. That constraint is a feature when a team has found a dependable way to fix a particular visual bug, build a component, tune a local server, or set up an unfamiliar repository. It is less useful when the real need is a broad procedure that should work without any reference session.

The project calls the successful original session the reference. The weaker or cheaper model that must repeat it is the student. The harness is the surrounding command-line agent that gives the model tools, files and instructions. Those terms matter because changing any one of them can change the result. A skill for one model inside omp is not automatically a skill for another model inside Codex.

The practical idea: do not preserve the final code as an answer key. Preserve the decisions, sequence, failure traps and checks that made the session work. Then see whether the intended student can follow them without hidden help.

How do you use Skill Distill after a good session?

The shortest path is to install the repository as a local skill, finish a task with Claude Code or Codex, then ask the teacher to distil that session for a named student. The repository gives separate symbolic-link installation paths for Claude Code and Codex, while the request itself can be plain language: identify the target harness, model and local server.

git clone https://github.com/sybil-solutions/skill-distill
# Then install the skill for Claude Code or Codex.

Before writing any reusable guidance, the procedure checks whether the student can answer a tiny task at all. That is an important separation. A missing API key, bad model identifier, unavailable server or broken adapter should be recorded as an environment failure, not blamed on the procedure being distilled.

Next comes reproducibility. The teacher extracts the original prompt verbatim, identifies the Git commit that existed when work began, and gives every trial a fresh copy of that starting point. The run instructions require a clean state; if the original began with uncommitted changes, they should first be captured on a temporary branch. This is where casual “save this as a skill” workflows often fail: a student inherits files or context that the reference happened to have.

What should the acceptance test measure?

The acceptance test is Skill Distill’s most useful contribution. Instead of judging a vague claim that the student “did the task,” the teacher writes a small list of observable outcomes that made the reference good. The framework suggests four to 10 checks, including executable checks where possible and visual evidence where scripts cannot decide quality.

A good acceptance list measures outcomes, not the teacher’s preferred route. For a web task, that could mean the page starts cleanly, a user action works, the changed area matches the intended layout, and the README explains how to run it. For a server task, it could mean the service launches, returns valid responses, stays inside a memory budget and retains the required context behaviour.

The reference must pass its own checks first. If it fails, the check is broken or incomplete. This rule stops the team from creating an impossible target for the student and forces an early distinction between a measurable requirement and an aesthetic impression.

Do not confuse feature parity with quality parity. The Skill Distill procedure warns that matching a thumbnail or a checklist can hide weaker typography, sparse visuals or poor interaction details. For visual work, compare the same screen area side by side at useful resolution and name the missing property precisely.

Why three trials instead of one?

Skill Distill uses three trials per version because agent runs are variable. Each trial receives the original prompt, a frozen copy of the current skill and a fresh starting state. It should not receive the teacher’s session history, the finished reference output or extra global instructions where the harness can disable them.

Three attempts do not prove a statistical guarantee. They do, however, expose a common failure pattern: a student that succeeds only when a tool call happens to land well, or only when an ambiguous instruction gets interpreted favourably. The point is operational confidence, not academic significance.

The repair method also matters. The teacher should identify the first divergence: the earliest wrong step that explains the final gap. If a student blocks its own terminal by starting a development server in the foreground, the fix belongs in the setup instruction. Adding more design advice later in the document will not repair that earlier failure.

Which harnesses can be students?

Skill Distill lists adapters for omp, pi, Codex, Claude Code and opencode. The project marks omp as tested. Codex is tested as a teacher, while its use as a student depends on provider arguments. Claude Code and pi are described as written adapters, and opencode is described as untested because its JSON log parser is not yet present.

That status list is more important than it looks. “The model can run” and “the distillation system can inspect, reset and judge its run” are different claims. If the harness cannot isolate context or provide readable logs, a successful output may be difficult to trust and hard to improve.

For a local open model, the likely first target is omp with a provider-backed model. For an Anthropic-compatible local server, the project says Claude Code needs an ANTHROPIC_BASE_URL route. The sensible first experiment is a task with a clear result and a fast check, not a week-long refactor with subjective acceptance criteria.

Home study setup with laptop showing coding notes on hollow square pattern
A dimly lit home study setup with laptop displaying coding notes on a hollow square pattern. · Photo by Altamash Mallick on Pexels

How does it differ from other skills?

Skill Distill belongs to the fast-growing SKILL.md ecosystem, but it solves a different problem from a library or a static engineering playbook. Antigravity Skills presents SKILL.md as a format for building and discovering agent skills; its public library includes packages for planning, frontend work, writing and coding discipline. That is distribution and discovery. Skill Distill is a post-session testing loop for making one new procedure credible on a specific student.

ApproachWhat is transferredHow it earns trustBest real-world useWhere it stops being useful
Skill DistillA successful coding session, rewritten as a student-specific operator skill.Fresh-state student runs, explicit acceptance checks and repeated repair.Replaying a proven task with a cheaper or local model.It stops helping when there is no good reference session, or when the target model and harness keep changing.
Matt Pocock skillsGeneral engineering practices such as planning, testing, review and project setup.Reusable, composable procedures maintained as a broad skill collection.Giving many agents a consistent engineering workflow.It stops at the point where a team needs proof that one weaker student can reproduce one exact prior result.
AdaSkill researchCapabilities and selected workflow guidance from multi-agent systems.Controlled interventions that test whether inherited structure helps or constrains the task.Deciding how much orchestration should survive conversion to one agent.It stops short of a ready-to-run coding-session replay tool with local harness adapters and user sign-off.
SkillFactoryModel behaviours such as reflection and retrying, converted into training traces.Supervised fine-tuning followed by reinforcement-learning workflows.Changing a model’s learned behaviour through training data.It stops being the practical answer when the team wants a reusable procedure today, without training a model.

Matt Pocock’s skills project remains a useful contrast. Its latest release notes show maintenance aimed at making agents read the actual SKILL.md, honour repository standards and use fresher subagent contexts. Those are broad safeguards for engineering agents. Skill Distill adds an evaluation harness around one extracted procedure, which is a more demanding but much narrower promise.

AdaSkill offers the clearest research warning against blindly copying workflows. The paper separates capability resources, such as tools and declarative knowledge, from pipeline guidance, such as ordering rules and decision gates. Its central finding is that the same guidance can improve one measure and worsen another. In plain English: preserve what gives the student new abilities, but be sceptical of rigid steps that may block a valid route.

SkillFactory is different again. It builds supervised fine-tuning data from reflection and verification traces, then describes reinforcement-learning training. That is weight-level behavioural change. Skill Distill stays outside the model weights: it changes the operating instructions, scripts and checks available at run time.

What makes a distilled skill portable?

Portability is not copying every word from the teacher’s transcript. A portable distilled skill identifies the essential decisions and leaves out dead ends, machine paths, secrets and context that only existed by accident. It should name commands, files, expected signals and recovery actions without turning the finished artifact into a hidden template.

The project’s line limit is useful discipline. Its guidance keeps the main SKILL.md under roughly 250 lines and moves detailed material into references that the student reads only when needed. This reduces the chance that a smaller model loses the actual sequence inside pages of background explanation.

Antigravity’s documentation makes the wider point: a skill starts as a draft that people review and edit, rather than an untouchable generated artifact. Its guide frames skills as reusable instruction packages. Skill Distill supplies the missing practical test: can the intended student use that package under controlled conditions

When should you use Skill Distill?

Use Skill Distill after a task genuinely went well and is likely to recur. Good candidates have a clear starting state, a stable prompt, observable success conditions and a student model chosen for a reason. A local model that is cheaper, more private or already available in a developer’s environment is an obvious target.

Start with work that can fail visibly. Examples include a repeatable frontend repair, a repository migration with tests, a particular component build, or a server configuration where latency and behaviour can be checked. The first run should teach the team whether the student’s limits are instruction-related or capability-related.

Do not use it to disguise an impossible substitution. If the target lacks the necessary tool access, context capacity, visual judgement or reasoning ability, longer instructions will not create those capabilities. The right result may be a documented boundary: keep the frontier model for this task, change the harness, or split the job into smaller stages.

What we could not verify?

Public repository material does not yet establish how often Skill Distill succeeds across different coding tasks, student models or hardware setups. It also does not publish a benchmark showing that a distilled skill transfers unchanged from one open-weight model to another. Sybil Solutions could settle those questions with a task suite, pinned student configurations, trial logs and independently repeatable acceptance results.

It is also not public whether the untested adapters will reach feature parity with the tested omp route, or how much manual judging a typical visual task requires. Teams considering production use should treat the first successful distillation as a local validation exercise, not a general performance claim.

Sources


Maya Chen
Maya Chen
Maya Chen covers AI agents, orchestration frameworks, tool-use, and evaluation. She focuses on what actually works in production—failure modes, safety boundaries, and measurable performance—without the hype.

Read more

Local News