Role Overview
Assess AI-assisted software development traces that support the training and evaluation of advanced AI models. You will review end-to-end coding sessions created with AI-enabled developer tools, evaluating technical correctness, workflow quality, and reasoning, then deliver clear written feedback using established rubrics.
Key Responsibilities
- Evaluate the quality and correctness of AI-assisted software development traces.
- Review multi-step coding workflows for soundness, correctness, and alignment with software engineering best practices.
- Assess code, debugging approaches, and reasoning across backend or full-stack systems.
- Provide concise, rubric-based written feedback on coding sessions.
Qualifications
- 3+ years of professional software development experience.
- Hands-on experience with AI-assisted coding tools and agentic or specification-driven workflows, such as Cursor, GitHub Copilot, Claude Code, or similar tools.
- Strong code-reading and debugging capabilities across full-stack or backend systems.
- Ability to evaluate multi-step coding trajectories for correctness and best practices.
Preferred Qualifications
- Experience with Kiro or Amazon CodeCatalyst.
- Prior experience evaluating or grading AI-generated code.
- Contributions to developer tooling.
Work Terms
- Remote role available to candidates located in the United States.
- Hourly engagement.
Compensation
$70 to $90 per hour.
Software Developer for AI Coding Trace Evaluation in charlotte at Unknown Company
This position is listed as full time and able to be worked remotely.