Survey Generator

Survey Generator

Provider-agnostic literature-survey artifact generator. Output flows into a pro-workflow wiki, not a standalone HTML file — survives sessions and indexes for FTS5 retrieval.

Diff vs dair-academy version

When to use

• "Survey on " / "lit review on "
• Onboarding a new domain — generate the map-of-the-field
• After a wiki has 10-30 sources, compile a synthesis page over them
• Pre-step before /wiki research runs: gives the loop a high-quality seed bundle

Inputs

Workflow (the agent runs these in order)

Step 1 — Read the anchor

WebFetch source_url. Extract subtopics + cited papers. For GitHub awesome-lists, walk README + linked papers files. For arXiv survey PDFs, use abstract + ToC.

Step 2 — Build research_bundle.json

Use templates/research_bundle.template.json as scaffold. Required keys:

jsonCopy code
{
  "topic": "...",
  "anchor_source": "...",
  "abstract_hints": ["..."],
  "taxonomy": [{"branch": "...", "children": [{"name": "...", "description": "..."}]}],
  "sections": [{"title": "...", "guidance": "...", "papers": ["key1","key2"]}],
  "bibliography": [{"key": "author-year-shortname", "authors": "...", "year": 2024, "title": "...", "venue": "...", "summary": "..."}]
}

Hard rules:

• Every paper in bibliography must be real. No invented entries.
• Every key referenced in sections[].papers must exist in bibliography.
• 4-8 taxonomy branches, 2-4 children each.
• 6-10 numbered sections covering: introduction → foundations → methods → evaluation → open problems.

Step 3 — Run the generator

bashCopy code
node $SKILL_ROOT/scripts/build-survey.js \
  --bundle <path-to-research_bundle.json> \
  --wiki <slug> \
  [--provider anthropic|openai|openrouter|fireworks|custom] \
  [--model <id>]

Generator:

1. Reads bundle.
2. Sends to LLM with strict markdown spec (numbered sections, inline [^paper-key] citations, no HTML).
3. Writes output to <wiki>/derived/surveys/<topic-slug>.md.
4. Appends bibliography rows to <wiki>/sources.md (deduped by key).
5. Calls wiki-cli.js page to upsert into FTS5 index.

Step 4 — Iterate

If prose is thin: tighten sections[].guidance and rerun. Output filename versions automatically (<slug>-v2.md, <slug>-v3.md).

To compare providers:

bashCopy code
node build-survey.js --bundle bundle.json --wiki agent-memory --provider openai --model gpt-4o
node build-survey.js --bundle bundle.json --wiki agent-memory --provider anthropic --model claude-opus-4-7

Each writes a separate versioned file; diff them.

Output structure

textCopy code
<wiki-root>/
├── sources.md                                 # bibliography rows appended (deduped)
└── derived/surveys/
    └── <topic-slug>-v1.md                     # the survey
        # title (h1)
        # ## 1. Introduction
        # ## 2. Foundations
        # ...
        # ## References
        # [^src-bib-<slug>] author year. title. venue.

Hard rules

1. Never invent bibliography entries — every paper must be a real work with venue.
2. Every section's papers array references keys in bibliography.
3. Output is markdown ONLY. No HTML, no inline SVG, no JS.
4. Bibliography rows in sources.md use the slug-style id src-bib-<slug> (derived from the bibliography key); cite as [^src-bib-<slug>]. Manual non-bibliography sources continue to use src-NNN.
5. Iterate on inputs (research_bundle.json), not on the generated output.
6. Provider+model selection is the user's call — never hardcode.

Composing with research loop

bashCopy code
/wiki init reasoning-models --title "Reasoning Models" --flavor research
# Manually compile a research_bundle.json
node skills/survey-generator/scripts/build-survey.js --bundle bundle.json --wiki reasoning-models
# Now the wiki has a structured survey + 50 bibliography rows
# Enable auto-research to expand:
# (edit reasoning-models/wiki.config.md, set auto_research.enabled: true)
node skills/wiki-research-loop/scripts/research-loop.js seed reasoning-models "chain-of-thought failure modes" --depth 0
node skills/wiki-research-loop/scripts/research-loop.js run reasoning-models