Factor Evaluation
Mining proposes factors; evaluation decides whether to believe them. This skill recomputes a library's metrics on a chosen split and exposes overfitting.
See references/metrics.md for precise metric definitions (IC vs. paper-IC, ICIR, redundancy correlation).
Workflow
1. Recompute metrics
bashfactorminer evaluate output/run1/factor_library.json \ --data path/to/market_data.csv \ --period test
--period selects the split: train, test, or both. Always lead with test — in-sample IC is not evidence.
2. Read the table
The output table reports, per factor: IC Mean, Paper IC, Abs IC, Paper ICIR, Win%, and Turnover. The summary block gives library-level means and the IC range.
3. Check decay
bashfactorminer evaluate output/run1/factor_library.json --data market_data.csv --period both
--period both adds a decay table (train Paper IC → test Paper IC → delta). A large negative delta is the signature of an overfit factor. Report decay honestly; do not quote the train number as the headline.
4. Rank the survivors
To shortlist the strongest signals only:
bashfactorminer evaluate output/run1/factor_library.json --data market_data.csv --period test --top-k 10
The top-K-by-IC table is the signal shortlist — the natural handoff to a research-idea workflow that wants to know which quantitative signals are currently working. The MCP screen_factors tool returns this same shortlist directly.
Interpreting the numbers
- IC ≈ 0.03–0.05 out of sample is a respectable single factor on liquid universes.
- ICIR matters more than IC: a small but stable IC beats a large erratic one.
- High turnover quietly erases IC once costs are applied — carry it into
factor-backtest.
Guardrails
- Never present
trainmetrics as the result. The deliverable is thetestnumber. - If every factor decays to ~0 on test, the library failed — say so. Do not search for a split that flatters it.
