Company: Handshake
Country: United States
Location: Remote
Job Type: Contract
Salary: USD100.00 - USD100.00
The AI Evaluator (Aviation & Transportation Operations) applies operational aviation expertise to evaluate, validate, and improve AI-generated content related to flight operations, air traffic coordination, dispatch planning, airfield operations, safety procedures, and regulatory compliance. This role ensures AI outputs are operationally realistic, safety-focused, and aligned with aviation standards.
The AI Evaluator (Aviation & Transportation Operations) applies operational aviation expertise to evaluate, validate, and improve AI-generated content related to flight operations, air traffic coordination, dispatch planning, airfield operations, safety procedures, and regulatory compliance. This role ensures AI outputs are operationally realistic, safety-focused, and aligned with aviation standards.
As an AI Evaluator, you will assess AI-generated responses, scenarios, simulations, and operational guidance involving aviation and transportation operations. You will identify procedural errors, safety risks, unrealistic assumptions, and regulatory misalignment, providing structured feedback to improve AI model accuracy and reliability in high-risk operational environments. Key responsibilities include: - 4+ years of working experience - Evaluating AI-generated content related to flight operations, dispatch, airfield management, and operational decision-making - Validating procedural accuracy against aviation regulations, SOPs, and safety management systems (SMS) - Assessing the realism of operational scenarios, risk assessments, and decision logic - Applying structured evaluation rubrics to score AI outputs and recommend improvements - Flagging safety-critical errors, regulatory non-compliance, or misleading operational guidance - Supporting continuous improvement of AI models through expert operational feedback
Application Deadline: 01 May, 2026
Job Status:
Submit Application