Lead Data Scientist, Data Products

• Define evaluation frameworks, offline and online metrics, and quality bars for internal models and ML products
• Build the measurement layer for cross-domain machine learning products
• Scale organizational visibility into performance across the business
• Own experimentation and monitoring for production models
• Contribute to monitoring key metrics and drift detection for enriched datasets
• Ensure quality for downstream athlete-facing experiences
• Identify and size AI/ML product opportunities
• Translate ambiguous problem spaces into scoped initiatives with defined success criteria
• Serve as the data science domain expert for the team
• Raise standards for baselines, validation, and evidence quality across ML engineers and cross-functional partners
• Partner closely with Machine Learning Engineers and cross-functional teams

Requirements

• 5+ years of experience in data science or a related quantitative domain
• Hands-on ownership of model evaluation and measurement for systems running in production
• Experience defining evaluation frameworks for ambiguous ML problems
• Experience with baseline selection, offline and online metric design, and validation strategies where ground truth is imperfect
• Strong SQL proficiency
• Comfort writing production-quality Python for statistical data processing
• Fluency in metrics and measurement for consumer software products
• Ability to translate business needs into technical plans and vice versa
• Ability to work on-site in the San Francisco office three days per week
• Must be located within commuting distance of San Francisco or willing to relocate
• Must address employment visa sponsorship status

Core Competencies

Demonstrates expertise in defining evaluation frameworks and metrics for machine learning products, with a strong focus on model evaluation, measurement, and data quality. Proficient in translating business needs into technical plans while collaborating effectively with cross-functional teams.

Highest-signal resume keywords

• Model Evaluation
• SQL Proficiency
• Python for Data Processing
• Evaluation Frameworks for ML
• Metrics and Measurement for Software

Hard Skills

• Model Evaluation
• Evaluation Frameworks
• Baseline Selection
• Metric Design
• Validation Strategies
• Statistical Data Processing
• Machine Learning
• Data Science
• Performance Monitoring
• Drift Detection

Soft Skills

• Collaboration
• Problem-Solving
• Communication

Industry Keywords

• Machine Learning Products
• Consumer Software
• Data Science Domain Expertise
• Cross-Functional Teams
• San Francisco Office