Healthcare AI Evaluation & Clinical Data Expert

Agents Only Technologies Inc. is building a specialized network of healthcare professionals and subject matter experts for an advanced healthcare data and AI evaluation project. The role involves creating healthcare scenarios and artifacts, developing complex prompts, establishing verified answers, creating evaluation rubrics, and assessing AI-generated responses for clinical accuracy, reasoning, completeness, and safety.

Responsibilities:

• Contributors will apply their professional expertise to create healthcare scenarios and artifacts, develop complex prompts, establish verified answers, create evaluation rubrics, and assess AI-generated responses for clinical accuracy, reasoning, completeness, and safety
• Project-specific onboarding, training, calibration, and qualification may be required

Requirements:

• Tier 3: 3+ years of relevant experience
• Tier 4: 5+ years of relevant professional or clinical experience
• Strong healthcare-domain knowledge
• Strong written English communication
• Excellent analytical and structured reasoning skills
• Ability to interpret complex healthcare information
• Strong attention to detail and quality
• Ability to work independently on complex assignments
• Physicians and senior clinicians
• Registered Nurses
• Pharmacists
• Nurse Practitioners / Advanced Practice Nurses
• Physician Assistants
• Clinical specialists and senior clinical reviewers
• Radiology and other specialized healthcare professionals
• Medical coding and billing specialists
• Claims and payer operations specialists
• Utilization management / prior authorization professionals
• Healthcare data and informatics specialists
• Clinical research professionals
• FHIR / healthcare data standards specialists
• Health information management professionals
• Other experienced healthcare subject matter experts
• Previous experience with AI evaluation, healthcare datasets, or AI systems