Location: Alexandria, VA
Compensation: $110,000 to $135,000 per year
Overview
This role exists to answer one question independently: do the synthetic-audience and automated systems actually perform as claimed? It is a verification and validation position focused on benchmarking system output against ground‑truth data to ensure accuracy, bias, and reliability.
Responsibilities
- Own the evaluation metrics, test harness design, and reported accuracy bounds; quantify system reliability and identify failure points.
- Certify fitness for use and translate results into actionable insights for stakeholders.
- Design and run independent validation methodologies, benchmarking synthetic-audience and automated outputs against real panel holdout data and other ground‑truth sources.
- Collaborate with cross‑functional teams to communicate findings and recommend system improvements.
Requirements
- Education & Experience : Bachelor’s degree in computer science, mathematics, engineering, or equivalent experience; Master’s degree preferred. Seven or more years of experience in data science, analytics, or a related field.
- Knowledge : Strong applied statistics and metrics design; fluency in accuracy, bias, calibration, and uncertainty quantification. Working understanding of large language models (LLMs) and related NLP technologies.
- Skills : Independence and rigor; ability to challenge results and withhold sign‑off when necessary. Clear communication; adept at explaining complex technical concepts to non‑technical audiences.
- Physical Requirements : Prolonged periods of sitting at a desk and working on a computer; frequent typing.
SHRM is an equal opportunity employer (Minority/Female/Disabled/Veteran). For more information, view our Statement of Accessibility.
#J-18808-Ljbffr