Automation QA Engineer with AI/LLM

Alpharetta, GA- Only Locals

6+ Months

Experience: 10+ years

Tech Stack: Python/JavaScript, Test Automation, Pytest/Selenium, LLM Evaluation, Jira/Test Management

Job Description:

Design and execute test cases for multi-turn agent conversations.
Build evaluation frameworks to assess LLM quality.
Automate API and UI testing.
Create regression suites for agentic outputs.
Drive test coverage and quality metrics.
Develop automated testing frameworks using Python/JavaScript, Pytest, Selenium, or similar technologies.
Validate AI/LLM responses and agent behavior against defined quality criteria.
Track defects and test results using Jira or other test-management tools.

Automation QA Engineer with AI/LLM_Alpharetta, GA- Onsite- Only Locals

Key skills