Machine Learning Task Auditor

Help strengthen the applied machine-learning tasks used to train and evaluate models at a frontier AI lab. You will assess task quality, correctness, and methodological rigor, with particular attention to experiment design, model-selection rationale, and evaluation methods. This is an applied and experimental ML evaluation role, not an LLM application development or MLOps position.

• Review applied machine-learning tasks for quality, correctness, and methodological soundness.
• Evaluate experiment design, model-selection reasoning, and evaluation methodology.
• Provide clear, written feedback using established rubrics.
• Assess ML claims against available evidence and reproduce results when needed.

• At least 3 years of hands-on applied or experimental machine-learning experience, including experiment design, model selection, hyperparameter tuning, and evaluation methodology.
• Strong data-quality rigor, including leakage detection, metric gaming, and appropriate train, test, and cross-validation practices.
• Proficiency with standard ML frameworks, including PyTorch, TensorFlow, scikit-learn, and XGBoost.
• Ability to critically evaluate machine-learning claims using evidence and reproduce results.

• Competition or benchmark experience, such as Kaggle.
• Graduate research experience or a publication record in applied machine learning.
• Previous task-grading or peer-review experience.

• Remote role open to candidates located in the United States.
• Hourly employment arrangement.

$70 to $90 per hour.