Data Science Expert
Comprehensive data science frameworks for analytics, machine learning, and data-driven decision making.
Data Strategy
Data Maturity Model
| Level | Name | Characteristics |
|---|
| 1 | Ad Hoc | Manual, inconsistent, siloed |
| 2 | Opportunistic | Some automation, point solutions |
| 3 | Systematic | Defined processes, governance emerging |
| 4 | Differentiating | Data-driven decisions, advanced analytics |
| 5 | Transformative | AI-first, competitive advantage |
Analytics Value Chain
DATA → INFORMATION → INSIGHT → ACTION → VALUE
PROGRESSION:
Descriptive: What happened?
Diagnostic: Why did it happen?
Predictive: What will happen?
Prescriptive: What should we do?
Autonomous: Self-optimizing systems
Statistical Analysis
Descriptive Statistics
CENTRAL TENDENCY:
- Mean: Sum / Count (sensitive to outliers)
- Median: Middle value (robust to outliers)
- Mode: Most frequent value
DISPERSION:
- Range: Max - Min
- Variance: Average squared deviation
- Standard Deviation: √Variance
- IQR: Q3 - Q1 (robust)
DISTRIBUTION SHAPE:
- Skewness: Asymmetry (0 = symmetric)
- Kurtosis: Tail heaviness (3 = normal)
For detailed inferential statistics and hypothesis testing, see Statistical Methods Reference.
Machine Learning
Algorithm Selection
| Task | Algorithms | When to Use |
|---|
| Classification | Logistic Regression, Random Forest, XGBoost, Neural Networks | Categorical outcomes |
| Regression | Linear Regression, Ridge/Lasso, Random Forest, XGBoost | Continuous outcomes |
| Clustering | K-Means, Hierarchical, DBSCAN | Group discovery |
| Dimensionality Reduction | PCA, t-SNE, UMAP | Feature reduction, visualization |
| Anomaly Detection | Isolation Forest, One-Class SVM, Autoencoders | Outlier detection |
| Time Series | ARIMA, Prophet, LSTM | Sequential data |
| Recommendation | Collaborative Filtering, Content-Based, Matrix Factorization | Personalization |
| NLP | Transformers, BERT, GPT | Text understanding/generation |
For detailed ML pipelines, feature engineering, and model monitoring, see ML Pipelines Reference.
Data Governance
Data Governance Framework
GOVERNANCE PILLARS:
POLICIES:
- Data ownership
- Data classification
- Data retention
- Data access
- Data quality standards
ROLES:
- Data Owner: Accountable for data domain
- Data Steward: Day-to-day quality management
- Data Custodian: Technical implementation
- Data Consumer: End user
PROCESSES:
- Data cataloging
- Metadata management
- Data lineage
- Issue resolution
- Change management
METRICS:
- Data quality scores
- Policy compliance
- Data access requests
- Issue resolution time
Data Quality Dimensions
| Dimension | Definition | Measurement |
|---|
| Accuracy | Correct representation of reality | % records matching source |
| Completeness | All required data present | % non-null values |
| Consistency | Same across systems | % matching across sources |
| Timeliness | Available when needed | Latency, freshness |
| Validity | Conforms to format/rules | % passing validation |
| Uniqueness | No unwanted duplicates | Duplicate rate |
Business Intelligence
BI Architecture
ARCHITECTURE LAYERS:
DATA SOURCES:
- Operational systems
- External data
- IoT/streaming
DATA INTEGRATION:
- ETL/ELT pipelines
- Data lakes
- Data warehouses
SEMANTIC LAYER:
- Business definitions
- Calculated metrics
- Hierarchies
- Relationships
PRESENTATION:
- Dashboards
- Reports
- Ad-hoc analysis
- Embedded analytics
Dashboard Design Principles
DESIGN PRINCIPLES:
PURPOSE:
- One clear objective per dashboard
- Know your audience
- Enable decisions
LAYOUT:
- Most important top-left
- Related items grouped
- Progressive disclosure
- Whitespace for clarity
VISUALS:
- Right chart for data type
- Consistent formatting
- Minimal decoration
- Color with purpose
INTERACTIVITY:
- Filters for exploration
- Drill-down capability
- Cross-filtering
- Tooltip details
Metric Design
METRIC DEFINITION TEMPLATE:
NAME: [Metric name]
DEFINITION: [Clear business definition]
FORMULA: [Precise calculation]
OWNER: [Responsible person]
DATA SOURCE: [Where it comes from]
GRAIN: [Level of detail]
FREQUENCY: [Update cadence]
DIMENSIONS: [Slicing attributes]
TARGETS: [Goals/benchmarks]
RELATED: [Related metrics]
Predictive Modeling
Use Case Framework
| Use Case | Business Application | Approach |
|---|
| Churn Prediction | Retention programs | Classification |
| Demand Forecasting | Inventory planning | Time series |
| Lead Scoring | Sales prioritization | Classification |
| Price Optimization | Revenue management | Regression/RL |
| Fraud Detection | Risk mitigation | Anomaly detection |
| Recommendation | Personalization | Collaborative filtering |
| Customer Segmentation | Marketing targeting | Clustering |
| Lifetime Value | Customer investment | Regression |
Data Ethics & Privacy
Ethical AI Framework
PRINCIPLES:
FAIRNESS:
- No discriminatory outcomes
- Bias testing across groups
- Regular auditing
ACCOUNTABILITY:
- Clear ownership
- Decision audit trails
- Escalation process
TRANSPARENCY:
- Explainable decisions
- Clear documentation
- User communication
PRIVACY:
- Data minimization
- Consent management
- Security controls
Bias Detection
BIAS TYPES:
HISTORICAL: Reflects past discrimination
REPRESENTATION: Training data not representative
MEASUREMENT: Proxy variables correlate with protected attributes
AGGREGATION: Single model for diverse populations
EVALUATION: Inappropriate benchmarks
FAIRNESS METRICS:
- Demographic Parity: Equal positive rates
- Equalized Odds: Equal TPR and FPR
- Individual Fairness: Similar inputs, similar outputs
- Calibration: Equal accuracy across groups
Analytics Team Structure
Team Roles
| Role | Focus | Skills |
|---|
| Data Engineer | Pipelines, infrastructure | SQL, Python, Spark, Cloud |
| Data Analyst | Reporting, ad-hoc analysis | SQL, BI tools, Statistics |
| Data Scientist | Modeling, ML | Python/R, ML, Statistics |
| ML Engineer | Model deployment | MLOps, Software Engineering |
| Analytics Engineer | Data modeling | dbt, SQL, Data Modeling |
Operating Models
| Model | Description | Best For |
|---|
| Centralized | Single analytics team | Consistency, efficiency |
| Decentralized | Embedded in business units | Business alignment |
| Hub & Spoke | Central CoE + embedded | Balance of both |
| Federated | Shared platform, domain teams | Scale with autonomy |
References
See Also
