Senior Data Labeling Jobs in Nashville (Remote) Rex.zone is hiring a Senior Data Labeling Specialist to own end-to-end data labeling and evaluation workflows for AI/ML training datasets across NLP, computer vision, and content safety. You will lead advanced annotation and reviewer-level QA, and run RLHF-style evaluations (preference ranking, rubric scoring, prompt evaluation) that connect label quality to model performance improvement.What You Will Do
- Label and review complex text, image, and multimodal datasets while maintaining high inter-annotator agreement.
- Run QA evaluation programs: sampling plans, error taxonomy, root-cause analysis, and corrective actions tied to guideline compliance.
- Perform RLHF evaluation workflows such as pairwise preference ranking, rubric-based scoring, and prompt evaluation for instruction-following and safety behavior.
- Execute NLP labeling tasks including named entity recognition, classification, summarization evaluation, and groundedness/faithfulness checks.
- Execute computer vision annotation such as bounding boxes, polygons, keypoints, segmentation, and attribute tagging; validate edge cases.
- Support content safety labeling (e.g., harassment, self-harm, hate) with consistent escalation paths and audit trails.
- Partner with engineering/ML teams to translate model failure modes into updated guidelines, new label schemas, and targeted re-annotation.
- Document decisions, dataset specs, and QA findings to support reproducibility and dataset lineage.
- 3+ years in data labeling/annotation or AI/ML evaluation, including QA evaluation and reviewer workflows.
- Hands-on experience with LLM evaluation, RLHF evaluation, or structured prompt evaluation using rubrics and preference judgments.
- Strong guideline compliance, error analysis, and understanding of how training data quality impacts model performance.
- Experience with NLP tasks (e.g., named entity recognition, text classification) and common dataset formats/tooling.
- Familiarity with computer vision annotation concepts and ambiguity resolution patterns.
- Ability to write clear guidelines, make consistent decisions, and communicate edge cases cross-functionally.
- Workplace: Remote, open to US-based candidates including Nashville, TN.
- May align to team collaboration hours for calibration and QA sign-off meetings.