Role Overview
Assess how well AI systems perform real-world accounting tasks by defining professional standards and evaluating completed work. Instead of preparing accounting deliverables, you will design detailed grading criteria and score AI-generated and human work samples, providing rigorous, written justifications for every judgment.
Key Responsibilities- Create precise, task-specific grading rubrics for common accounting deliverables, including reconciliations, month-end close documentation, financial statements and disclosures, audit-ready workpapers, and technical accounting memos.
- Score AI-generated and human-prepared work samples according to those criteria, producing clear, evidence-based written explanations for each score.
- Apply consistent, reproducible judgment so scores are defensible and can be calibrated across reviewers.
- Receive and incorporate structured feedback from senior reviewers, and iterate quickly to improve criteria and scoring consistency.
- Minimum 5 years of professional accounting experience.
- Experience in audit, assurance, or advisory at major global accounting firms, or senior accounting roles at large public companies.
- CPA or equivalent professional certification strongly preferred.
- Deep, practical knowledge of US GAAP application, month-end close processes, account reconciliations, preparation of financial statements and disclosures, and maintaining audit-ready workpapers.
- Exceptional written communication, able to explain precisely why work does or does not meet professional standards.
- Highly detail-oriented, consistent in judgment, and comfortable having evaluations reviewed and calibrated against peers.
- Prior experience with AI training, evaluation, or human-data labeling projects is a strong plus.
- Remote role.
- Employment type: hourly.
- 130 - 160 hourly.
To apply, submit your resume or a summary of relevant professional experience. Candidates who meet the qualifications may be invited to complete a brief technical assessment or provide additional information upon request.