Continuous Learning
Summary
Continuous Learning skill domain covering knowledge operations.
Tools: High, Medium, Low, Negative, Attack Patterns, Defense Patterns, Tool Behaviors, Environment Patterns (+1 more)
Domain: knowledge
Use Cases
- Post-Engagement Learning — After a penetration test, extract reusable patterns from observations and tool outputs
- Cross-Session Knowledge Building — Accumulate tool behavior knowledge across multiple engagements into structured entries
- Pattern Recognition — Identify recurring vulnerabilities, tool limitations, and effective technique combinations
- Knowledge Confidence Tracking — Assign and update confidence scores for learned facts based on corroboration frequency
- Memory Layering — Distill raw observations into layered knowledge: immediate notes → verified patterns → core principles
Activation
- After completing a penetration test engagement
- When encountering a novel attack technique or defense
- After a tool produces unexpected results
- When identifying recurring patterns across targets
- User says "learn", "remember this", "pattern", "lesson learned"
Learning Cycle
┌──────────┐ ┌───────────┐ ┌───────────┐ │ Pattern │───→│ Extract & │───→│ Confidence │ │ Detection│ │ Structure │ │ Scoring │ └──────────┘ └───────────┘ └───────────┘ │ ┌──────────┐ ┌───────────┐ ┌───────┴───┐ │ Cross- │←──│ Memory │←──│ Storage │ │ Reference│ │ Retrieval │ │ │ └──────────┘ └───────────┘ └───────────┘
Step 1: Pattern Detection
Identify learnable patterns from observations:
| Pattern Category | What to Look For |
|---|---|
| Attack Patterns | Successful exploitation techniques, unexpected bypasses |
| Defense Patterns | WAF rules detected, IDS triggers, rate limiting behaviors |
| Tool Behaviors | Tools that produced false positives, missed findings, gave unusual output |
| Environment Patterns | Common misconfigurations, typical network architectures |
| Engagement Patterns | Time allocation, methodology gaps, scope surprises |
Step 2: Extract & Structure
Transform raw observation into structured knowledge:
markdown## Knowledge Entry: [ID] - **Category:** [Attack / Defense / Tool / Environment / Engagement] - **Context:** [When/where this was observed] - **Pattern:** [What happened] - **Root Cause:** [Why it happened, if known] - **Applicability:** [When this knowledge is relevant] - **Source:** [Engagement type, tool, or research] - **Date:** [When observed]
Step 3: Confidence Scoring
Rate confidence based on supporting evidence:
| Confidence Level | Criteria | Storage Action |
|---|---|---|
| High | Observed 3+ times across different engagements, independently verified | Store as established pattern |
| Medium | Observed 1-2 times, consistent with known theory | Store as provisional pattern |
| Low | Single observation, unclear root cause | Store as observation only |
| Negative | Previously held belief contradicted by evidence | Flag old entry for review, store correction |
Step 4: Storage
Save structured knowledge to the appropriate memory layer:
Short-term (Engagement): Tactical observations for current engagement
- Specific tool configurations that worked
- Target-specific quirks discovered
- Time-critical findings to revisit
Medium-term (Technique): Technique-level knowledge reusable across engagements
- Effective payload patterns for specific technologies
- Bypass techniques for specific WAF/IDS products
- Tool-specific tips and gotchas
Long-term (Strategic): Strategic patterns about security landscapes
- Emerging attack technique trends
- Common architectural weaknesses by industry
- Tool capability boundaries and gaps
Step 5: Cross-Reference
Link new knowledge to existing entries:
- Does this contradict a previous observation?
- Does this reinforce a previous observation?
- Does this fill a gap in existing knowledge?
- Is this a variant of a known pattern?
Knowledge Categories
Attack Pattern Library
markdown### [Pattern Name] - **Technique:** [ATT&CK technique ID if applicable] - **Prerequisites:** [What must be true for this to work] - **Steps:** [How to execute] - **Indicators of success:** [How to know it worked] - **Variations observed:** [Different contexts where it applied] - **Counter-indications:** [When NOT to attempt this]
Tool Mastery Notes
markdown### [Tool Name] - [Specific Use Case] - **Command:** [Exact command with flags] - **Context:** [When this configuration is optimal] - **Output interpretation:** [How to read the results] - **Gotchas:** [Common mistakes or misleading output] - **Alternatives:** [Other tools for the same job] - **Performance notes:** [Speed/resource considerations]
Engagement Lessons
markdown### Lesson: [Title] - **Engagement type:** [Black box / White box / Red team / Bug bounty] - **What happened:** [Description] - **What we learned:** [Key takeaway] - **What we'd do differently:** [Process improvement] - **Applicable scenarios:** [When this lesson is relevant]
Learning Triggers
Automatic pattern extraction should occur when:
- An exploit succeeds unexpectedly — Why did it work? What was different?
- A tool gives unexpected output — Is this a false positive, a real finding, or a tool bug?
- A target behaves unusually — Custom defense? Misconfiguration? Honeypot?
- An attack chain works particularly well — What made the chain effective?
- A previously reliable technique fails — Has the target been patched? Is there a new defense?
- Time is wasted on a dead end — What signal was missed that could have prevented this?
Integration with Other Skills
| Skill | Learning Opportunity |
|---|---|
verification-loop | Learn which verification methods are most reliable per finding type |
terminal-ops | Learn optimal evidence capture formats per engagement type |
deep-research | Learn which sources are most authoritative per topic |
search-first | Learn which repositories have the best exploits per technology |
vulnerability-assessment | Learn scanner false positive patterns per target technology |
web-sqli / web-xss | Learn payload patterns effective against specific WAFs |
network-pentest | Learn network architecture patterns and their common weaknesses |
security-bounty-hunter | Learn which vulnerability types are most rewarded per platform |
Knowledge Quality Rules
- Never store assumptions as facts — Label speculation clearly
- Always include context — Where, when, and under what conditions
- Prefer specific over general — "SQL injection in parameter X of WordPress plugin Y v3.2" beats "SQL injection exists"
- Include negative results — "Tool X missed finding Y" is as valuable as "Tool X found Y"
- Date everything — Knowledge has a shelf life; dated entries can be refreshed
- Source attribution — Every entry must trace back to a specific observation or research
Detection Methods
Learning Pipeline Anomalies
- Training data injection: Sudden spike in training data volume from untrusted source.
- Model weight drift: Model weights changing unexpectedly between training cycles.
- Backdoor activation: Specific input patterns triggering anomalous predictions.
RAG / Knowledge Base Indicators
- Vector store poisoning: Documents with embedding far from cluster centroid.
- Retrieval anomalies: Specific queries consistently retrieving same documents (potential trigger).
- Cross-user data leakage: User A's query retrieving User B's indexed documents.
SIEM Detection Rules
- Splunk SPL:
index=ml training_run=* | stats dc(data_source) by model_version | where dc > 5 - LangSmith / Helicone: Anomaly detection on RAG retrieval patterns.
Defense Evasion Techniques
Data Poisoning Stealth
- Slow injection: Add poisoned samples over multiple training cycles; below distribution shift threshold.
- Mimic legitimate distribution: Match poisoned samples' statistics to legitimate data.
- Trigger-based backdoor: Activates only on specific input patterns; otherwise benign.
- Label-flipping attacks: Flip labels of small % of samples; degrades model accuracy subtly.
RAG Poisoning Stealth
- Match embedding distribution: Poisoned documents close to legitimate cluster centroid.
- Trigger via specific query: Backdoor activates only on attacker's specific query pattern.
- Cross-tenant indexing: Index poison in tenant A; affect queries from tenant B (if not isolated).
Memory Poisoning Stealth
- Long-term memory manipulation: Modify agent memory to plant triggers for future sessions.
- Cache poisoning: Poison LLM gateway cache; affects future users.
- Gradual trust building: Build up reputation over time, then exploit at scale.
Anti-Patterns
- Storing unverified findings — Verify before storing (use verification-loop)
- Over-generalizing — One observation does not make a universal rule
- Ignoring context — A technique that worked in one context may fail in another
- Knowledge hoarding — Store structured patterns, not raw data dumps
- Never pruning — Review and update stored knowledge periodically
- Confidence inflation — Be honest about confidence levels
Orchestration
ECC Loop Pattern
- Pattern: Learning Cycle (detect pattern → extract & structure → score confidence → store → cross-reference)
- Rationale: Learning is inherently iterative — each observation refines understanding, confidence scores evolve with more evidence, and cross-referencing with existing knowledge prevents contradictions
- Integration: All skills (consumes observations from every engagement), verification-loop (learns which verification methods are most reliable), search-first (learns which sources have best results per topic), terminal-ops (learns optimal evidence formats)
Cross-Skill Pipeline
[all skills] → continuous-learning → MEMORY.md / chronicle ↓ ↑ verification-loop (FP patterns) deep-research (context enrichment)
Quality Gate
- Pre-condition: Observation from real engagement or verified research
- Post-condition: Structured knowledge entry with confidence level and source attribution
- Verification: Entry cross-referenced against existing knowledge, no contradictions
