Job Title6+ years building evaluation or testing infrastructure for ML/LLM systems or large-scale distributed systemsProven ability to translate product requirements into measurable metrics and test plans.Strong Python skillsStrong experience with modern data toolingHands-on experience running A/B tests, canaries, or experiment frameworks.Experience defining and maintaining operational reliability metrics (SLIs/SLOs) for AI-driven systems.Familiarity with large-scale distributed or streaming systems serving AI/agent workflows (millions of events or alerts/day).Excellent communication skills -- able to clearly convey technical results and trade-offs to engineer, PMs, and analysts.