Implement and refine testing workflows tailored for AI powered solutions where outputs are probabilistic rather than deterministic.
Build guardrails to identify model drift, prompt regression, hallucination risks, and safety violations before they hit production.
Own the creation, maintenance, and scaling of high-quality golden datasets used as the source of truth for benchmarking model changes.
Champion programmatic evaluation tools such as Promptfoo, DeepEval, or Ragas to automate the assessment of prompt variants and model performance.
Partner closely with the Chapter Lead, QA to translate high-level AI QA frameworks into actionable, daily tasks for technical teams.
Provide real-time feedback to the Chapter Lead on how the strategy is performing in the trenches, recommending practical pivots based on real-world results.
Collaborate with DevOps specialists to inject AI evaluation metrics directly into modern CI/CD pipelines, ensuring automated gates handle fluid AI responses.
Validate data pipelines feeding our models, ensuring that data integrity is maintained from ingestion to inference.

College diploma or Degree in Computer Science, Engineering, or a related discipline (or equivalent practical experience).
A strong foundation in core QA principles combined with a passion for AI/ML.
3+ years of experience in Quality Assurance, Software Engineering, or Data Engineering.
Experience in Python or JavaScript/TypeScript (essential for configuring evaluation frameworks like Promptfoo and manipulating test data).
Experience with test automation frameworks, API testing, and code repositories (Git).

Flexibility: Yes, we think that what you do matters. At work and at home.
Fun: We laugh a lot, it makes every day brighter.
Discounted services: We provide amazing services to our clients, and you’ll get them at home, because you deserve them.
Rewarding Pay: Let's be honest, everybody likes to make a good salary. We offer attractive compensation packages, and it comes with a great culture.
Benefits: We’ve got you covered.
Career Evolution: Join us and we will give you the tools to achieve your career goals!
Technology: You have a passion for technology? Excellent, we do too. Here, you will manage, influence, play, create, fix, and shape the industry.

QA Test Automation Specialist

Key skills