Role Overview
Review AI-assisted software development sessions that support the training and evaluation of advanced AI models. You will assess end-to-end coding traces for technical correctness, sound workflows, and reasoning quality, then deliver clear written feedback using established rubrics.
Key Responsibilities
- Evaluate the quality and correctness of AI-assisted software-development traces.
- Review complete coding sessions produced with AI-enabled developer tools.
- Assess code correctness, development workflow quality, reasoning, and alignment with best practices.
- Provide clear, rubric-based written feedback.
Qualifications
- 3+ years of professional software development experience.
- Hands-on experience with AI-assisted coding tools and agentic or specification-driven workflows, such as Cursor, GitHub Copilot, Claude Code, or similar tools.
- Strong code-reading and debugging skills across full-stack or backend systems.
- Ability to evaluate multi-step coding trajectories for correctness and best practices.
Preferred Qualifications
- Experience with Kiro or Amazon CodeCatalyst.
- Previous experience evaluating or grading AI-generated code.
- Contributions to developer tooling.
Work Terms
- Remote role, available to candidates located in the United States.
- Hourly engagement.
Compensation
$70 to $90 per hour.
Software Developer for AI Coding Trace Evaluation in Harrisburg at Unknown Company
This position is listed as full time and able to be worked remotely.