Diogenes

A senior-reviewer audit for AI-slop: a verdict, an attributable findings table, structural metrics, and a human rewrite.

Problem

"This sounds like AI" is a useful instinct and a useless review. A writer cannot act on a vibe. The tells of machine-generated prose are specific (definitional throat-clearing, stacked tricolons, vague superlatives, a vocabulary of delve and leverage and seamless), and a review is only actionable when it points at the passage and names the pattern.

Diogenes reads submitted writing the way its namesake read Athens: skeptical of polished surfaces, loyal to material specifics, unimpressed by definitions that flatter the reader. It returns a verdict, a findings table tied to the actual text, structural metrics, and a from-scratch human rewrite.

Architecture

One skill backed by one subagent, authored to the agentskills.io specification. The design separates three concerns so each can change independently. 

/diogenes:audit <text | file | url>  <audience + medium>
        │
        ▼  context: fork  (clean context, no upstream conversation)
[diogenes subagent]  model: sonnet, read-only tools + WebFetch
        │
        ├─ skills/audit/SKILL.md          six-category framework + output spec
        └─ references/research.md          vetted citations, read on demand
        │
        ▼
VERDICT · TAKEAWAYS · FINDINGS TABLE · STRUCTURAL OBSERVATIONS
· AUTHENTICITY IMPROVEMENTS · HUMAN REWRITE · CONFIDENCE

The skill uses context: fork with an agent: reference, so every audit runs in a clean subagent context with no access to the upstream conversation, immune to bias from whatever was being discussed before. The subagent is pinned to sonnet with read-only tools plus WebFetch, which lets it read a published page directly when the submission is a URL.

Findings cite a fixed set of three peer-reviewed papers (Juzek & Ward 2025, Muñoz-Ortiz 2024, Reinhart et al. 2025), read from a bundled reference file when the auditor needs to ground a pattern. Citations outside that set are forbidden, which keeps the audit honest instead of inventing authority.

Evaluation

The audit ships an evals.json that forks the Engram evaluation framework (same LLM-as-judge method, same claude --bare isolation). Because the audit is read-only, the filesystem-snapshot machinery is dropped; each scenario materializes a fixture holding a submission.md and grades the report on two axes.

The first axis is structure: every required section is present (Verdict, Takeaways, Findings table with its exact columns, Structural observations, Authenticity improvements, Human rewrite, Confidence). The second is label accuracy: the emitted verdict is compared against the fixture's expected label, and an adjacent-but-wrong label is a failure. Fixtures span all three labels the skill emits: ai-slop, human-voice, and ai-assisted-but-natural. Grading the labels across all three is what keeps the auditor from defaulting to "this is AI" on clean prose.

Three learnings

  1. 1. Attributable findings beat a global score. A number ("80% likely AI") gives the writer nothing to fix. A table that quotes the actual tell and names the category turns the review into edits, which is the only output a writer can use.
  2. 2. A clean fork removes the reviewer's bias. Running the audit in a forked context with no access to the prior conversation means it judges the text in front of it, not the discussion that produced it. An auditor that saw the drafting would grade the intent, not the result.
  3. 3. Test the false positive, not just the catch. The hardest fixture is the clean human post that uses a few polished words: an over-eager detector flags it as slop. Grading a human-voice label as strictly as an ai-slop label is what keeps the tool from crying wolf on competent writing.

Links