Kg Builder

KG Builder

A knowledge graph is a product with a schema, not a pile of triples. Quality comes from pipeline order: model the domain before extracting, validate during extraction, fuse before storing, and attach provenance to every fact from the first write.

This skill covers the full build — value test, ontology, extraction, quality gate, entity resolution, serving, and maintenance — plus the boundary question that decides whether the result is trustworthy: which stages are deterministic code and which are LLM judgment.

Scope note. This is about knowledge graphs — what an agent remembers. It is not about task graphs, agent orchestration, or multi-agent topology.

Reference Files

The deterministic / LLM boundary

Decide this before writing code. Code owns control flow, identity, validation, and merges. The model gets contained judgments behind a typed interface, each with a measured baseline.

Measure every LLM surface against a prompt-only baseline before trusting it. This is not theoretical caution. In a pre-registered real-model evaluation of an LLM-adjudicated dedup and contradiction loop, the loop trailed a plain prompt-only baseline by 0.28–0.33 on detection and safety across every provider cell tested. Adjudication that is not measured is decoration.

Workflow

Phase 1 — Design (do not skip)

1. Value test. A graph pays off when queries are multi-hop ("who worked with X on projects using Y"), when entities recur across documents, or when the relationships are the data. If every query is a single-hop lookup or an aggregation, use a table and stop here. Write the kill criterion down before continuing.
2. Competency questions. Write the 10–20 questions the graph must answer. These are the ontology's spec and its test suite. Anything you cannot path through the finished schema is a missing type or relation.
3. Ontology. 5–15 entity types, 10–30 relation types, each relation with explicit domain and range. Precise verb names (ACQUIRED, DEPENDS_ON) — never RELATED_TO. Keep it in ontology.yaml as the single source of truth; every extraction prompt embeds it verbatim.
4. Storage and identity. Choose property graph (default), RDF/OWL (interop, description-logic reasoning), or typed edges in SQLite (<50K nodes). Decide now how time and provenance attach to every fact — retrofitting provenance after fusion is effectively impossible.

Validate the schema before extracting anything:

bashCopy code
uv run scripts/validate_ontology.py ontology.yaml

Phase 2 — Extract

5. Route by source type. Structured sources (databases, CSVs, APIs) map column → type in deterministic code with no model involved. Semi-structured sources (HTML tables, infoboxes) get per-layout parsers. Only unstructured text enters the LLM pipeline. Running NLP over already-structured data is the classic waste.
6. Entities. Dictionaries and exact rules first for closed vocabularies — free, deterministic, perfect precision. LLM extraction for open text, with the ontology in the prompt. Always capture surface form, canonical guess, type, source pointer, and confidence.
7. Relations. Extract only between entities that already passed step 6; never let relation extraction invent endpoints. Constrain output to the ontology's relation list and validate domain/range in code. Require a verbatim evidence quote that asserts the relation — co-occurrence is not assertion ("Musk discussed Twitter" is not OWNS).
8. Events. For dynamic domains, extract events as first-class nodes (trigger + typed arguments + time), never flattened into pairwise edges — flattening loses which acquisition happened at which price.

Phase 3 — Consolidate

9. Quality gate. Sample 50 items and score entity precision and relation precision before fusing anything. Target ≥90% precision. Fix the prompt or the rules, then re-run — never hand-patch the output. Recall improves with more passes; bad precision poisons the graph permanently.
10. Fusion. Blocking → matching → merge. Blocking avoids O(n²) comparison; matching scores string, attribute, and structural evidence (two J. Smith nodes sharing three coauthors and an affiliation are one person; identical names with disjoint neighborhoods are not); merge policy is deterministic code that keeps the canonical name, unions aliases and edges, preserves conflicting values with provenance, and records merged_from for undo. An erroneous merge is far more damaging than a missed one — it silently fuses two entities' entire edge sets. Auto-merge only above the high band; queue the middle for review.

Check the blocking strategy against labeled pairs before running it at scale:

bashCopy code
uv run scripts/blocking_report.py candidates.jsonl --labels matches.jsonl

Phase 4 — Serve and maintain

11. Serving. Entity-link the query, expand 1–2 hops, serialize the subgraph as compact (head)-[REL {time, source}]->(tail) lines grouped by head. For multi-hop questions retrieve paths between the query's entities, not neighborhoods around each — the path is the answer skeleton. Cluster and pre-summarize for "what are the themes" questions.
12. Maintenance. New facts supersede rather than overwrite: keep the prior claim with status, supersedes, and validity interval. When a new fact contradicts a stored one, keep both with time and provenance and prefer the newer at retrieval. Re-run fusion periodically — unmaintained memory graphs rot exactly like unfused extractions.

Output

Deliver these artifacts, in this order:

Report precision as a sampled estimate with its sample size. A precision number without a stated sampling method is a vibe.

Working rules

• Schema first, always. Extraction without an ontology produces a word cloud with arrows. If the user resists schema design, induce a minimal 5-type ontology from three sample documents and show it for approval — never skip to extraction.
• Provenance on every fact. source, extracted_at, confidence. Non-negotiable; fusion and trust both depend on it.
• Pilot before scale. Run 10 documents through all four phases first. The pilot exposes ontology gaps at 1% of the cost.
• Never auto-accept an induced schema. LLM-proposed ontologies overfit their sample documents. Prune to the minimal set that answers the competency questions.
• The LLM is stage machinery, not the pipeline. It slots into extraction and the ambiguous matching band. The surrounding schema, validation, and merge policy are what make the output a knowledge graph rather than a transcript.

Errors and troubleshooting