Quality Engineer

Evolent partners with health plans and providers to improve outcomes for people with complex and costly health conditions. The Quality Engineer develops and executes exploratory and automated tests for the Evolent Platform web user experience, maintains test automation, manages defects, and reports on quality. The role also applies AI-assisted engineering tools while ensuring security, privacy, regulatory, and Responsible AI compliance.

Responsibilities:

• Develop, document, and execute test plans and test cases for assigned features
• Perform exploratory and automated testing to ensure software functionality, reliability, and performance
• Conduct regression testing to ensure new code does not adversely impact existing functionality
• Collaborate with development teams to understand requirements, expected agent behavior, and human-in-the-loop flows, and identify potential quality issues early
• Participate in code reviews, design discussions, and sprint planning, and contribute to the design and implementation of enhancements to the team’s test systems
• Create and maintain Playwright end-to-end tests and Vitest component tests using the team’s established frameworks and patterns
• Execute and maintain contract tests and API tests using the team’s established tooling
• Test streaming, loading, and error states, including slow and failed agent responses
• Execute automated test suites in GitHub Actions, evaluate results, and fix failing scripts to maintain a high pass rate
• Create and maintain synthetic test data
• Follow and help reinforce the team’s testing processes and automation standards, holding peers accountable constructively and making sure the team understands the risks of exceptions
• Run automated axe-core accessibility checks and manual screen reader tests (JAWS, NVDA, VoiceOver) against WCAG 2.1/2.2 AA
• Identify, isolate, document, and track defects, working closely with developers to ensure timely resolution
• Analyze test and evaluation results, generate quality metrics, and report on software quality status to the team and stakeholders
• Present work in Agile ceremonies, participate in knowledge-sharing sessions, and seek feedback to continuously improve skills
• Utilize approved AI-assisted engineering tools to accelerate test case design, test script development, test data generation, documentation, and troubleshooting
• Evaluate and validate AI-generated tests, scripts, and analysis for correctness, security, maintainability, compliance, and business appropriateness before use
• Learn and adopt the prompts, skills, agents, and AI-assisted testing patterns defined by the team and share improvements that benefit the team
• Identify opportunities to improve testing productivity and quality through AI-enabled workflows and automation
• Follow enterprise policies governing AI usage, intellectual property protection, security, privacy, PHI handling, and Responsible AI practices
• Ensure no confidential, regulated, or protected information (including PHI in test data) is used with unapproved AI services or tooling; use synthetic data for testing
• Participate in peer reviews and validation processes for AI-assisted solutions

Requirements:

• 3+ years of experience in software quality assurance or testing, including hands-on UI test automation
• Experience with Playwright or similar end-to-end automation tools
• Working knowledge of JavaScript or TypeScript for test automation
• Familiarity with component testing frameworks such as Vitest or Jest
• Familiarity with API testing tools (e.g., Postman) and REST APIs
• Familiarity with CI/CD tools (e.g., GitHub Actions) and version control systems such as Git
• Familiarity with test management and defect tracking tools (e.g., JIRA, ALM)
• Proficiency in software testing methodologies, including functional, regression, and performance testing
• Knowledge of Agile methodologies and experience working in an Agile development environment
• Strong analytical skills and attention to detail, with a focus on troubleshooting and problem-solving
• Excellent communication skills, openness to feedback, and the ability to collaborate effectively with cross-functional teams
• Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related field, or equivalent experience
• preferably using TypeScript or JavaScript
• Exposure to testing AI or LLM output using test datasets and scoring rubrics
• Familiarity with testing React applications and streaming user interfaces
• Familiarity with contract testing and basic load testing
• Working knowledge of accessibility testing, including axe-core and screen readers (JAWS, NVDA, or VoiceOver)
• Exposure to AI-assisted testing tools such as GitHub Copilot or Playwright MCP
• Familiarity with FHIR R4
• Experience in healthcare or other regulated environments handling PHI