McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. The Software Automation Engineer is responsible for designing, developing, and maintaining automated test solutions across Ontada’s product ecosystem, with a strong focus on AI enabled and GenAI powered systems.
Responsibilities:
- Own and execute test strategy, planning, and execution for assigned features, services, or product areas under the guidance of the QA Lead
- Identify functional, integration, and nonfunctional quality risks early; communicate risks, impacts, and recommendations clearly
- Author comprehensive test strategies, test plans, and test cases aligned with product requirements and acceptance criteria
- Perform exploratory testing to uncover complex, edge case, and systemic defects
- Coordinate end to end validation across multiple environments to ensure release readiness
- Design, develop, and maintain automated test suites across UI, API, service, and data layers
- Contribute to the enhancement and maintainability of automation frameworks using tools such as Selenium, Playwright, Cypress, TOSCA, or similar
- Develop robust API automation using RestAssured, Postman, or equivalent frameworks
- Implement effective test data strategies, including synthetic data generation and environment setup
- Integrate automated tests into CI/CD pipelines to support fast and reliable feedback cycles
- Leverage AI assisted development tools (e.g., GitHub Copilot, Claude Code, or similar) to accelerate test automation development, refactoring, and debugging while maintaining code quality and security standards
- Use AI tooling to assist with test case generation, edge case identification, and data driven scenario expansion, validating all outputs through engineering judgment and established QA practices
- Design and execute test strategies for AI/ML and GenAI powered features, including LLM based workflows
- Validate prompt behavior, prompt templates, and prompt chaining across different scenarios and data contexts
- Perform negative testing for AI systems, including prompt injection, jailbreak attempts, hallucination risks, and unsafe outputs
- Test Retrieval Augmented Generation (RAG) pipelines, including:
- Embedding quality validation
- Retrieval accuracy, recall, and relevance
- Chunking and indexing strategies
- Validate AI outputs for accuracy, consistency, explainability, and compliance in regulated environments
- Collaborate with Engineering and ML Ops teams to test model integrations, configuration changes, and inference pipelines
- Utilize AI powered tools to support prompt analysis, test scenario exploration, and hypothesis generation when validating LLM based features and AI workflows
- Critically evaluate AI generated suggestions and outputs to ensure accuracy, safety, reproducibility, and regulatory compliance
- Perform advanced backend testing across SQL and NoSQL data systems
- Validate data ingestion, transformations, persistence, and integrity across services and environments
- Coordinate testing of asynchronous workflows and integrations (e.g., message queues, APIs, batch processes)
- Work closely with Product Owners and Business Analysts to refine user stories, define acceptance criteria, and ensure testability
- Partner with developers during design and implementation to support shift left testing
- Participate actively in sprint planning, grooming, retrospectives, and release readiness reviews
- Collaborate with onshore and offshore QA team members to ensure consistent execution and quality standards
- Ensure testing activities align with HIPAA and other regulatory, security, and data privacy requirements
- Contribute audit ready documentation, including test plans, execution evidence, and reports
- Participate in root cause analysis for quality or performance issues and support corrective actions
- Identify opportunities for improving QA processes, tools, and documentation; contribute suggestions through established continuous improvement channels
- Research and evaluate new QA, automation, or performance testing, AI assisted tools and techniques as appropriate
Requirements:
- Degree or equivalent and typically requires 4+ years of relevant experience
- Bachelor's degree in computer science, Engineering, Mathematics, or equivalent practical experience
- 4+ years of progressive Software Quality Assurance experience, preferably in healthcare or regulated industries
- 3+ years of hands-on test automation development experience
- 2+ years of API testing and automation experience
- 3+ years of backend testing experience using SQL and/or NoSQL databases
- 3+ years of software performance testing experience, including test planning, execution, and analysis
- 1+ years of experience testing AI/ML or GenAI systems, or demonstrated delivery of AI adjacent quality frameworks (e.g., prompt testing, RAG evaluation, guardrails)
- Experience owning QA execution for complex product areas with limited day to day oversight
- Experience mentoring or supporting junior QA engineers
- Strong experience working in Agile SDLC environments with CI/CD pipelines
- Proficiency in Java, JavaScript, or Python for test automation and scripting
- Experience with CI/CD tools such as Jenkins, GitHub Actions, GitLab CI and build tools like Maven or Gradle
- Solid understanding of QA methodologies, test design techniques, and quality metrics
- Hands-on experience with performance testing tools (JMeter, NeoLoad, or similar)
- Experience using profiling and monitoring tools (Dynatrace, New Relic, AppDynamics, Splunk, JProfiler)
- Ability to analyze performance issues related to CPU, memory/heap, garbage collection, threads, databases, messaging systems, and network latency
- Experience creating reusable, maintainable, and portable automation and performance test scripts
- RAG testing experience, including embedding quality, retrieval evaluation, and chunk strategy validation
- Familiarity with vector databases and semantic search concepts
- Hands-on experience using AI assisted coding and analysis tools such as GitHub Copilot, Claude Code, or similar
- Ability to apply AI tools effectively for test automation development and refactoring, debugging and root cause investigation, exploratory test design and edge case discovery
- Strong understanding of limitations and risks of AI generated outputs, with the ability to validate, correct, and harden results for production quality use
- Experience with source control tools such as GitHub, Bitbucket, Git Bash
- Experience with test management tools (qTest, TestRail, ALM, TestLink, or similar)
- Familiarity with microservices and distributed system architectures
- Experience benchmarking, capacity planning, and release readiness reporting
- Knowledge of healthcare software, data privacy, and regulatory compliance is a plus
- Ability to manage multiple priorities and work independently in a fast paced environment