Generalist | $40/hr Remote | Mercor

Generalist Evaluator Expert

Organization: Mercor

Referral Partner: Crossing Hurdles

Type: Hourly Contract

Compensation: $35–$40/hour

Location: Remote

Duration: Approximately 1 month

Commitment: 10-40 hours/week, flexible and asynchronous

Role Responsibilities

• Design and optimize detailed prompts with multiple constraints and clear instructions for language model evaluation
• Define and document evaluation standards and develop comprehensive rubrics for assessing correct responses in consumer contexts
• Conduct testing and grading by running prompts through AI models and assessing outputs against expectations
• Support benchmarking and quality assurance to ensure prompt rigor and consistency before integration into official evaluation sets
• Collaborate with team members in quality assurance review processes to maintain evaluation reliability

Requirements

• BS or BA degree from a reputable institution (completed or in progress)
• Strong writing skills, critical thinking, and instructional clarity
• Ability to work independently and meet deadlines
• Significant familiarity with ChatGPT or similar language tools for personal or professional use
• Experience in teaching or research is preferred
• Access to a desktop or laptop computer (Chromebooks are not supported)

Application Process

• Upload resume
• AI interview based on your resume (15 min)

Seniority Level: Mid-Senior level

Employment Type: Contract

Job Function: Quality Assurance, Writing/Editing, and Research

Industries: E-Learning Providers, Education, and Information Services