Company: Handshake
Country: United States
Location: remote
Job Type: Remote
Salary: USD0.00 - USD0.00
The AI Evaluator (Generalist) supports the evaluation and improvement of AI-generated content across a wide range of everyday business, operational, and customer-facing scenarios. This role focuses on clarity, logic, tone, completeness, and real-world usability, helping ensure AI outputs are accurate, understandable, and appropriate for general audiences.
As an AI Evaluator, you will review and assess AI-generated responses, prompts, and scenarios across multiple domains, including customer service, administrative tasks, basic operations, and general knowledge. You will identify clarity issues, logical gaps, tone mismatches, and usability concerns, and provide structured feedback to improve AI performance and consistency. Key responsibilities include: - Evaluating AI-generated content for clarity, coherence, and completeness - Assessing logical flow, relevance, and appropriateness of responses - Applying structured evaluation rubrics to score AI outputs and suggest improvements - Identifying ambiguity, missing context, or misleading information - Providing clear, constructive written feedback to improve AI response quality - Supporting quality assurance across diverse, non-specialized content areas
- Bachelor’s degree in any field - Strong reading comprehension and analytical thinking skills - Ability to evaluate written content objectively and follow detailed evaluation guidelines - Clear written communication and attention to detail - Comfort working remotely and reviewing written or scenario-based content - Reliable judgment and consistency when applying evaluation criteria Preferred (but not required): - Experience in administrative, customer service, operations, or content review roles - Familiarity with rubrics, quality assurance, or review processes - Interest in AI, digital tools, or content evaluation Compensation Per completed evaluation task (rate may vary based on task scope and complexity)
Application Deadline: 30 May, 2026
Job Status:
Submit Application