We are seeking a detail-oriented and analytical Annotation Data Engineer to support annotation quality, machine learning data operations, and dataset validation for AI/ML initiatives. The ideal candidate will have experience in annotation quality management, data quality engineering, and quality measurement systems, preferably within physical AI, robotics, computer vision, or foundation model development.
Key Responsibilities
* Define, implement, and maintain annotation quality standards, guidelines, defect taxonomies, and acceptance criteria.
* Develop and maintain quality metrics and reporting frameworks to measure accuracy, consistency, completeness, rework rates, throughput, and reviewer agreement.
* Design and execute sampling strategies, annotation audits, escalation workflows, and root-cause analysis processes for large-scale annotation programs.
* Evaluate generated captions, reasoning outputs, and foundation model training datasets against established quality benchmarks.
* Analyze annotation quality trends, identify recurring defects, and coordinate corrective and preventive actions with cross-functional teams.
* Validate dataset releases to ensure compliance with defined quality requirements and readiness for model training.
* Maintain traceable quality records, audit documentation, issue logs, and continuous improvement plans.
* Conduct calibration sessions and support reviewer alignment to improve annotation consistency and inter-annotator agreement.
* Collaborate with annotation operations, data engineering, quality engineering, and AI/ML teams to optimize annotation workflows.
* Leverage SQL, Python, and reporting tools to analyze quality data, automate measurements, and generate actionable insights.
Required Technical and Functional Skills
* 25 years of relevant experience in annotation quality, machine learning data operations, quality engineering, or data quality management.
* Experience supporting physical AI, robotics, computer vision, or machine learning programs.
* Demonstrated experience developing quality metrics, KPIs, and performance measurement systems.
* Hands-on experience working with large-scale annotation programs and dataset quality assurance processes.
* Experience evaluating generated captions, reasoning outputs, or foundation model training datasets.
* Strong understanding of annotation guidelines, sampling plans, acceptance criteria, defect classification, and audit methodologies.
* Ability to perform quality trend analysis, identify root causes, and drive corrective actions.
* Working knowledge of SQL and Python for data analysis, quality measurement, and reporting.
* Experience creating quality dashboards, reports, and performance tracking mechanisms.
* Strong analytical, problem-solving, documentation, and cross-functional communication skills.
by Jobble
Key Responsibilities
* Define, implement, and maintain annotation quality standards, guidelines, defect taxonomies, and acceptance criteria.
* Develop and maintain quality metrics and reporting frameworks to measure accuracy, consistency, completeness, rework rates, throughput, and reviewer agreement.
* Design and execute sampling strategies, annotation audits, escalation workflows, and root-cause analysis processes for large-scale annotation programs.
* Evaluate generated captions, reasoning outputs, and foundation model training datasets against established quality benchmarks.
* Analyze annotation quality trends, identify recurring defects, and coordinate corrective and preventive actions with cross-functional teams.
* Validate dataset releases to ensure compliance with defined quality requirements and readiness for model training.
* Maintain traceable quality records, audit documentation, issue logs, and continuous improvement plans.
* Conduct calibration sessions and support reviewer alignment to improve annotation consistency and inter-annotator agreement.
* Collaborate with annotation operations, data engineering, quality engineering, and AI/ML teams to optimize annotation workflows.
* Leverage SQL, Python, and reporting tools to analyze quality data, automate measurements, and generate actionable insights.
Required Technical and Functional Skills
* 25 years of relevant experience in annotation quality, machine learning data operations, quality engineering, or data quality management.
* Experience supporting physical AI, robotics, computer vision, or machine learning programs.
* Demonstrated experience developing quality metrics, KPIs, and performance measurement systems.
* Hands-on experience working with large-scale annotation programs and dataset quality assurance processes.
* Experience evaluating generated captions, reasoning outputs, or foundation model training datasets.
* Strong understanding of annotation guidelines, sampling plans, acceptance criteria, defect classification, and audit methodologies.
* Ability to perform quality trend analysis, identify root causes, and drive corrective actions.
* Working knowledge of SQL and Python for data analysis, quality measurement, and reporting.
* Experience creating quality dashboards, reports, and performance tracking mechanisms.
* Strong analytical, problem-solving, documentation, and cross-functional communication skills.
by Jobble
Annotation Data Engineer in dallas at Unknown Company
This position is listed as full time and onsite.