Idea Scoring

Skill: idea-scoring

Purpose

Produce a single, defensible verdict on an idea by aggregating all available dimension scores using a multiplicative-floor algorithm — a single catastrophic weakness kills the score, just like it kills a real startup. Includes a Riskiest Assumption Test (RAT) to convert the verdict into a concrete next action.

Input

• Idea slug
• One or more of the following (uses whatever is available):
  • memory/ideas/<slug>/idea.md
  • memory/ideas/<slug>/desire_scores.json
  • memory/ideas/<slug>/competitors.json
  • memory/ideas/<slug>/pricing.json
  • memory/ideas/<slug>/cac.json
  • memory/ideas/<slug>/market_size.json
  • memory/ideas/<slug>/distribution.json
  • memory/ideas/<slug>/retention.json
  • memory/ideas/<slug>/complexity.json
  • memory/ideas/<slug>/weighted_signals.json
• Optional: memory/user_profile.md (for founder-market fit)

Minimum Viable Input

At least 3 of 7 dimensions must have source data. If fewer are available, refuse to score and list what's missing. Two dimensions are mandatory — Demand and Distribution. Without evidence of a real problem and a path to reach users, scoring is meaningless.

Scoring Dimensions

Dimension Score Mapping

Each dimension maps source data to a 0–100 sub-score using the rubrics below. When source data uses qualitative labels, apply these conversions.

Demand (0–100)

Adjust within range: +10 if cross-platform resonance confirmed, +5 if monetization_validated is true in idea.md.

Competition (0–100)

Higher = more favorable competitive landscape (counterintuitive — think of it as "opportunity score").

Adjust: +10 if top competitor complaints reveal an unserved pain point. -15 if a FAANG-class player owns the category.

Monetization (0–100)

Adjust: +10 if freemium_conversion_estimate > 5%. +5 if market_size_verdict = "large".

Distribution (0–100)

Adjust: +10 if founder has existing audience or distribution edge (from user_profile.md).

Retention (0–100)

Adjust: +5 if natural_usage_frequency is daily. -10 if weekly-or-less with no external trigger.

Founder-Market Fit (0–100)

If user_profile.md is unavailable, default to 50 (neutral) and flag as missing.

Scoring Algorithm

Step 1 — Compute dimension sub-scores

Apply the mapping rubrics above. Record each as d_i (0–100).

Step 2 — Apply floor penalty (the "Killer Dimension" rule)

Any dimension scoring below 25 is a potential startup killer. Apply this penalty:

Copy code
floor_penalty = 1.0
for each dimension d_i:
    if d_i < 25:
        floor_penalty *= (d_i / 25)

This multiplicative penalty means a single catastrophic weakness (score 0–10) can halve or destroy the final score, regardless of how strong other dimensions are. This reflects startup reality: brilliant distribution cannot save a product nobody wants.

Step 3 — Compute weighted base score

Copy code
weights = {
    demand: 0.20,
    competition: 0.10,
    monetization: 0.20,
    distribution: 0.20,
    retention: 0.15,
    founder_market_fit: 0.15
}

base_score = sum(d_i * w_i for each dimension)

Step 4 — Apply floor penalty and missing-input discount

Copy code
missing_discount = available_dimensions / total_dimensions
adjusted_score = base_score * floor_penalty * missing_discount
final_score = round(clamp(adjusted_score, 0, 100))

The missing-input discount ensures that ideas scored on only 3 of 7 dimensions can never reach the "pursue" tier without completing more analysis.

Step 5 — Determine confidence level

Step 6 — Issue verdict

Riskiest Assumption Test (RAT)

Every idea rests on assumptions. The RAT identifies the single assumption that, if wrong, kills the idea — and designs the cheapest possible experiment to test it before building anything.

RAT Identification Process

1. List all assumptions embedded in the idea (drawn from dimension scores and source data):

   • Demand: "People actually have this problem and will seek a solution"
   • Monetization: "Users will pay $X/mo for this"
   • Distribution: "We can reach users via [channel] at acceptable cost"
   • Retention: "Users will come back [frequency]"
   • Competition: "Our differentiator matters to users"
   • Founder fit: "I can build this with my current skills/resources" (using user_profile.md if available)

2. Rank by two axes (each 1–5):

   • Criticality: If wrong, how dead is the idea? (5 = instant kill)
   • Uncertainty: How little evidence do we have? (5 = pure speculation)

3. RAT = assumption with highest (criticality × uncertainty). Ties broken by criticality.

RAT Experiment Design

For the identified RAT, design an experiment following these constraints:

Experiment types by assumption category

Pass/Fail Threshold

Define the threshold before running the experiment. The threshold is written into scores.json so it can be evaluated later. Thresholds must be:

• Specific: a number, not "good engagement"
• Time-bound: measured within the experiment window
• Binary: pass or fail, no "sort of passed"

Process (step by step)

1. Load all available dimension files from memory/ideas/<slug>/.
2. Check minimum viable input (≥ 3 dimensions, including Demand and Distribution). If not met, refuse and list missing inputs.
3. Map each dimension's source data to a 0–100 sub-score using the rubrics.
4. Compute floor_penalty from any sub-scores below 25.
5. Compute base_score using weighted sum.
6. Apply floor_penalty and missing_discount to get final_score.
7. Determine score_confidence.
8. Issue verdict from threshold table.
9. Identify top_strengths (top 2 dimensions) and top_weaknesses (bottom 2 dimensions).
10. Run RAT identification: list assumptions, score criticality × uncertainty, select the riskiest.
11. Design RAT experiment with pass/fail threshold.
12. If this is a pivot re-score, write to pivot_scores.json instead.

Output

Write to memory/ideas/<slug>/scores.json (or pivot_scores.json for re-scores):

jsonCopy code
{
  "dimension_scores": {
    "demand": 0,
    "competition": 0,
    "monetization": 0,
    "distribution": 0,
    "retention": 0,
    "founder_market_fit": 0
  },
  "weights_applied": {
    "demand": 0.20,
    "competition": 0.10,
    "monetization": 0.20,
    "distribution": 0.20,
    "retention": 0.15,
    "founder_market_fit": 0.15
  },
  "floor_penalty": 1.0,
  "base_score": 0,
  "missing_discount": 1.0,
  "final_score": 0,
  "verdict": "pursue | test | pivot | drop",
  "score_confidence": "high | medium | low",
  "missing_inputs": [],
  "top_strengths": [
    { "dimension": "", "score": 0, "reason": "" }
  ],
  "top_weaknesses": [
    { "dimension": "", "score": 0, "reason": "" }
  ],
  "killer_dimensions": [],
  "riskiest_assumption_test": {
    "assumption": "",
    "category": "demand | monetization | distribution | retention | competition | founder_fit",
    "criticality": 0,
    "uncertainty": 0,
    "rat_score": 0,
    "experiment": {
      "type": "",
      "description": "",
      "duration": "",
      "estimated_cost": "",
      "pass_threshold": "",
      "fail_action": "pivot | drop | re-test with different channel"
    },
    "all_assumptions_ranked": [
      { "assumption": "", "criticality": 0, "uncertainty": 0, "rat_score": 0 }
    ]
  }
}

Notes

• Re-scoring pivots: When scoring a pivot variant from pivot_options.json, write output to memory/ideas/<slug>/pivot_scores.json. Include a pivot_id field referencing the option.
• Score decay: If source data is older than 90 days (check analyzed_at or file timestamps), apply a 10% confidence penalty and flag stale inputs.
• Geometric vs. additive: The floor penalty provides multiplicative dynamics (one zero kills the score) while the weighted sum provides interpretable dimension contributions. This hybrid outperforms pure additive (hides fatal flaws) and pure geometric (too punishing for moderate weaknesses).