Skip to main content
Status: implemented (designed 2026-07-18) · Skill: hypothesis-exploration / paper-exploration

​
Context

Both generate skills run the same small state machine over a markdown backlog table, and both currently leave the mechanics unimplemented. Interim (until the module is implemented): each skill orchestrates the row edits by hand / via direct tool calls; once honest-scholar is installed (via ensure-tooling) the skill calls honest-scholar backlog … instead. The two skills:
  • hypothesis-exploration (../../../skills/hypothesis-exploration/SKILL.md, §Verbs, § Tooling) drives rows in a paper’s docs/research/<paper>/backlog.md.
  • paper-exploration (../../../skills/paper-exploration/SKILL.md, §Verbs) drives rows in docs/research/portfolio-backlog.md, and on promote also writes the paper registry docs/research/papers.md.
They are the same shape one level apart (ADR-0005 three-level mirror; ADR-0006 two-skills-per-level, ../../../decisions/0006-two-skills-per-level.md): identical verbs, identical firewall, identical file-drawer discipline. Editing the table by hand is fragile — provenance snippets are verbatim (may contain |), illegal state transitions are easy to make, and drop reasons get skipped. One shared helper removes the drift risk and makes both TODOs disappear at once.

​
Goal

A single, level-parametric backlog helper that appends and transitions rows through parked → candidate → ranked → promoted | dropped, while preserving provenance and recording drop reasons. It performs the mechanical file operations the two skills already specify; it makes no scientific judgment and never selects what to promote.

​
Design sketch

One module, two column profiles. A single module, honest_scholar/exploration/backlog.py (exposed as honest-scholar backlog, shared by both exploration skills), parameterized by --level {hypothesis|paper}. The row state machine, provenance rules, and drop discipline are identical across levels; only the scored columns and the promote target differ. Row schema (shared columns in bold, level-specific in italics):
  • id / one-line / provenance / status / note are shared and load-bearing.
  • EIG and frame (gap-spotting vs. problematization) exist only at the hypothesis level; the paper level ranks on feasibility × interest alone.
Transition verbs (invoked as honest-scholar backlog <verb>; each validates the source state and refuses illegal moves): park and add never rank; promote and drop are terminal and never delete a row (file-drawer discipline — ../01-lifecycle.md §3). A read-only list companion (honest-scholar backlog list [--level L] [--status S]) prints matching rows and mutates nothing — the everyday inspection command, the backlog analog of progress status. It performs no transition, so it sits outside the transition table above. Provenance preservation. The helper copies the origin verbatim and never paraphrases. For a scout-seeded row it stores the source-paper-id plus the citing-context snippet returned by literature scout (grounding: ../../../resources/references/citation-scouting.md); for EDA, eda:<dataset-id> + observation; for hand-parked, own. Because snippets can contain table-breaking characters, the helper is responsible for escaping on write and round-tripping on read (see Open questions on the storage format). promote scaffolds and links the next-stage artifact. This is the only verb that touches files outside the backlog, and only on an explicit human pick:
  • hypothesis level → create docs/research/<paper>/hypotheses/<YYYY-MM-DD-slug>/, write hypothesis.md from the shared template with the status frontmatter block (../../../resources/templates/; frontmatter contract in ../../../skills/progress/SKILL.md), carry the provenance forward, then set the row to promoted with a relative link to the folder. Hand-off to hypothesis-testing.
  • paper level → scaffold the paper root (hypotheses/, backlog.md, paper/), seed paper/pitch.md from the row, and append the registry row to docs/research/papers.md (paper-id → root + backend: binding). A backlog candidate is not in papers.md until promoted — the backlog is proposals, the registry is committed papers. Hand-off to paper-synthesis.
Interaction with papers.md. The helper only ever appends a registry row on paper-level promote; it never edits an existing registry row and never sets a decision.md verdict. progress (../../../skills/progress/SKILL.md) reads the registry + artifact frontmatter read-only (not the backlog table) and is unchanged by this proposal.

​
Dependencies & posture

  • Light-dep: pyyaml + stdlib only (matches literature’s honest_scholar/literature/graph.py and the substrate). pyyaml covers the status frontmatter it writes on promote and the papers.md row; the backlog table itself is plain markdown.
  • Firewall — proposes only. Every verb but promote stays inside exploration; promote runs only on an explicit human pick and is the sole path out. The helper mechanically scaffolds and links — it never ranks by fiat, never auto-promotes the top row, and never writes a verdict (meta-spec §2.1, §2.3, ../00-meta-spec.md).
  • Interim path stays valid. Until the module lands, both skills keep editing the table by hand in the documented column order; the honest-scholar backlog command is additive.

​
Open questions

  1. Storage format. Keep the human-readable markdown table and have the helper escape/round-trip provenance, or back the backlog with a YAML sidecar rendered to markdown for reading? The table is author-friendly and already parsed by rank/list; a sidecar is safer for verbatim snippets. Leaning: keep the table, escape on write.
  2. Slug ownership. Does promote mint the <YYYY-MM-DD-slug> / paper-id, or does the human supply it? (paper-id must be stable and never reused.)
  3. Concurrency. Two sessions appending to one backlog — is a simple append-only + git-merge posture enough, or is a lock needed?

​
Acceptance criteria

  • One helper serves both levels via --level, with the two column profiles above.
  • All five verbs implemented; illegal transitions rejected with a clear message.
  • park/add refuse a row without provenance; drop refuses without a reason and retires (never deletes) the row.
  • Scout provenance (source-paper-id + citing-context snippet) is stored verbatim and survives a read/write round-trip unaltered.
  • promote scaffolds the correct next-stage artifact, writes its status frontmatter, links it from the row, and — at paper level — appends the papers.md registry row; it runs only on an explicit id argument (human pick).
  • No verb writes a verdict or decision.md; progress continues to read the registry + artifact frontmatter unchanged (it does not read the backlog table).
  • Deps limited to pyyaml + stdlib.
  • ../../../skills/hypothesis-exploration/SKILL.md — verbs, row schema, the TODO.
  • ../../../skills/paper-exploration/SKILL.md — portfolio backlog, papers.md.
  • ../../../skills/progress/SKILL.md — status frontmatter this helper writes.
  • ../00-meta-spec.md §2.1/§2.3 — agency + firewall.
  • ../01-lifecycle.md §3 — file-drawer / rigor kit.
  • ../../../decisions/0006-two-skills-per-level.md — two-skills-per-level.
  • ../../../resources/references/citation-scouting.md — provenance grounding.