Engram

Filesystem-backed agent memory: consolidation, session briefing, and working state, with a reproducible eval.

Problem

An agent loses its working state every time the context window compacts: what it was doing, what it decided, what is still open. Native Claude Code memory covers two boundaries well, implicit learnings (auto memory) and an idle-session recap, but it leaves gaps: auditing the knowledge directories a project accumulates, graduating a stable learning into a permanent rule, orienting from across tools at session start, and holding structured in-flight work through a restart.

Engram packages three skills that operate at exactly those boundaries, and defers everywhere native memory already does the job.

Architecture

Three skills, one plugin, all authored to the agentskills.io specification. State is plain files, so it survives the runtime and stays auditable. 

consolidate   audit knowledge dirs (docs/, staging, archive) + auto memory
              → editable plan file → executes only approved actions
              graduates stable feedback memories to rules / CLAUDE.md

briefing      read-only session-start orientation, aggregating
              issue tracker + git history + staleness signals

working       explicit .memory/ for in-flight work (todos, decisions)
              lifecycle: checkpoint → promote (to ADR / tracker) → cleanup

consolidate scans declared knowledge directories, generates a plan file the user reviews and edits, then executes only the approved actions; it treats native MEMORY.md as read-only and focuses on graduation and on directories outside auto memory's scope. briefing is read-only and complements the native recap by pulling cross-tool signals on demand. working manages an explicit .memory/ directory with a three-command lifecycle: checkpoint to capture, promote to move state into permanent homes such as ADRs or tracker entries, cleanup to clear after promotion.

Evaluation

Each skill ships an evals.json following the agentskills.io evaluation spec, extended with filesystem-state assertions because these skills produce side effects. The runner forks the Advisors evaluation framework: same LLM-as-judge methodology, same claude --bare clean-room isolation, same git-shorthash provenance. Before each run it materializes a fixture tree into a temp workspace, and grading checks both the text output and the filesystem state left behind.

Recorded result

Nine scenarios across the three skills, each run with the skill and against a bare-prompt baseline. Mean pass rate was 90% with the skill versus 77% without, a 13-point gain. The lift concentrates where the skill adds structure (an explicit checkpoint format, division of labor with auto memory, staleness signals folded into the briefing); scenarios where the baseline already produces good output show no gap, which the table records rather than hides.

The run model was claude-sonnet-4-6. Eighteen runs plus eighteen grading calls cost $1.07 and finished in about five minutes. Keeping the eval cheap is what makes it a habit rather than a ceremony.

Evaluation runner, fixtures, and method (README)

Three learnings

  1. 1. Build beside the native feature, not over it. Auto memory and the idle recap already do their jobs well; duplicating them would be churn. Engram earns its place by covering the boundaries native does not, and explicitly defers everywhere native is sufficient.
  2. 2. Grade the side effect, not just the text. These skills write files, so a text-only judge would miss the point. Extending the eval with filesystem-state assertions, materialized from a fixture tree per run, is what lets the score reflect whether the skill actually produced the right artifact on disk.
  3. 3. Report the zero-deltas honestly. Several scenarios show no gain because the baseline already handles them, and the table says so. A skill eval that only showed its wins would not tell me where the skill is load-bearing versus decorative, and that is the information I actually act on.

Links