Monitor the performance of Agentic Capabilities as they autonomously process various ticket types
Ensure seamless integration of AI agents into new or existing workflows, optimizing for efficiency and accuracy
Review fallout tickets (cases where AI agents cannot resolve issues) within a workflow management tool
Diagnose root causes, make necessary corrections, and re-ingest updated information to the AI system
Analyze fallout patterns to identify knowledge gaps, process inefficiencies, or opportunities for AI improvement
Develop and implement training protocols to enhance Agentic Capabilities, leveraging prompt engineering, model validation, and knowledge base updates
Collaborate with cross-functional teams (product, engineering, support) to align AI behaviors with business needs and compliance requirements
Maintain and update the agent knowledge base, ensuring accurate, current, and comprehensive content for AI agents
Validate AI performance through systematic review, testing, and user/stakeholder feedback
Track and report on key metrics: ticket resolution rates, fallout frequency, review turnaround times, and AI improvement outcomes

Bachelor’s degree (BS/BA) desired
2+ years of related experience
Understanding of the business function and M&Ps; align AI behavior with policy and process; plus knowledge of supported workflows and tools with experience operating the systems involved
Strong analytical and problem-solving skills with meticulous attention to detail; proven root cause analysis capability
General AI literacy and understanding of agentic systems; basic prompt engineering (iteration, testing, versioning)
Ability to manage fallout within SLAs, triage tickets, and drive rapid resolution; strong prioritization in fast-paced environments
Observability: proficiency with logs, metrics, dashboards, and alerts; define and track quality KPIs (accuracy, fallout rate, MTTR)
Basic scripting understanding to automate corrections, content re-ingestion, and validation workflows
Knowledge base authoring and maintenance; clear documentation of training methods, resolutions, and changes for auditability
Compliance/data privacy/ethical guidelines awareness; maintain auditable processes and change logs
Effective communication: synthesize findings, report metrics, and present recommendations to stakeholders

Medical/Dental/Vision coverage
401(k) plan
Tuition reimbursement program
Paid Time Off and Holidays (at least 23 days of vacation each year and 9 company-designated holidays)
Paid Parental Leave
Paid Caregiver Leave
Additional sick leave beyond what state and local law require may be available but is unprotected
Adoption Reimbursement
Disability Benefits (short term and long term)
Life and Accidental Death Insurance
Supplemental benefit programs: critical illness/accident hospital indemnity/group legal
Employee Assistance Programs (EAP)
Extensive employee wellness programs
Employee discounts up to 50% off on eligible AT&T mobility plans and accessories, AT&T internet (and fiber where available) and AT&T phone

Senior Specialist, Quality – AI Training Manager

Key skills