Software Automation Engineer

McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. The Software Automation Engineer is responsible for designing, developing, and maintaining automated test solutions across Ontada’s product ecosystem, with a strong focus on AI enabled and GenAI powered systems.

Responsibilities:

• Own and execute test strategy, planning, and execution for assigned features, services, or product areas under the guidance of the QA Lead
• Identify functional, integration, and nonfunctional quality risks early; communicate risks, impacts, and recommendations clearly
• Author comprehensive test strategies, test plans, and test cases aligned with product requirements and acceptance criteria
• Perform exploratory testing to uncover complex, edge case, and systemic defects
• Coordinate end to end validation across multiple environments to ensure release readiness
• Design, develop, and maintain automated test suites across UI, API, service, and data layers
• Contribute to the enhancement and maintainability of automation frameworks using tools such as Selenium, Playwright, Cypress, TOSCA, or similar
• Develop robust API automation using RestAssured, Postman, or equivalent frameworks
• Implement effective test data strategies, including synthetic data generation and environment setup
• Integrate automated tests into CI/CD pipelines to support fast and reliable feedback cycles
• Leverage AI assisted development tools (e.g., GitHub Copilot, Claude Code, or similar) to accelerate test automation development, refactoring, and debugging while maintaining code quality and security standards
• Use AI tooling to assist with test case generation, edge case identification, and data driven scenario expansion, validating all outputs through engineering judgment and established QA practices
• Design and execute test strategies for AI/ML and GenAI powered features, including LLM based workflows
• Validate prompt behavior, prompt templates, and prompt chaining across different scenarios and data contexts
• Perform negative testing for AI systems, including prompt injection, jailbreak attempts, hallucination risks, and unsafe outputs
• Test Retrieval Augmented Generation (RAG) pipelines, including:
• Embedding quality validation
• Retrieval accuracy, recall, and relevance
• Chunking and indexing strategies
• Validate AI outputs for accuracy, consistency, explainability, and compliance in regulated environments
• Collaborate with Engineering and ML Ops teams to test model integrations, configuration changes, and inference pipelines
• Utilize AI powered tools to support prompt analysis, test scenario exploration, and hypothesis generation when validating LLM based features and AI workflows
• Critically evaluate AI generated suggestions and outputs to ensure accuracy, safety, reproducibility, and regulatory compliance
• Perform advanced backend testing across SQL and NoSQL data systems
• Validate data ingestion, transformations, persistence, and integrity across services and environments
• Coordinate testing of asynchronous workflows and integrations (e.g., message queues, APIs, batch processes)
• Work closely with Product Owners and Business Analysts to refine user stories, define acceptance criteria, and ensure testability
• Partner with developers during design and implementation to support shift left testing
• Participate actively in sprint planning, grooming, retrospectives, and release readiness reviews
• Collaborate with onshore and offshore QA team members to ensure consistent execution and quality standards
• Ensure testing activities align with HIPAA and other regulatory, security, and data privacy requirements
• Contribute audit ready documentation, including test plans, execution evidence, and reports
• Participate in root cause analysis for quality or performance issues and support corrective actions
• Identify opportunities for improving QA processes, tools, and documentation; contribute suggestions through established continuous improvement channels
• Research and evaluate new QA, automation, or performance testing, AI assisted tools and techniques as appropriate

Requirements:

• Degree or equivalent and typically requires 4+ years of relevant experience
• Bachelor's degree in computer science, Engineering, Mathematics, or equivalent practical experience
• 4+ years of progressive Software Quality Assurance experience, preferably in healthcare or regulated industries
• 3+ years of hands-on test automation development experience
• 2+ years of API testing and automation experience
• 3+ years of backend testing experience using SQL and/or NoSQL databases
• 3+ years of software performance testing experience, including test planning, execution, and analysis
• 1+ years of experience testing AI/ML or GenAI systems, or demonstrated delivery of AI adjacent quality frameworks (e.g., prompt testing, RAG evaluation, guardrails)
• Experience owning QA execution for complex product areas with limited day to day oversight
• Experience mentoring or supporting junior QA engineers
• Strong experience working in Agile SDLC environments with CI/CD pipelines
• Proficiency in Java, JavaScript, or Python for test automation and scripting
• Experience with CI/CD tools such as Jenkins, GitHub Actions, GitLab CI and build tools like Maven or Gradle
• Solid understanding of QA methodologies, test design techniques, and quality metrics
• Hands-on experience with performance testing tools (JMeter, NeoLoad, or similar)
• Experience using profiling and monitoring tools (Dynatrace, New Relic, AppDynamics, Splunk, JProfiler)
• Ability to analyze performance issues related to CPU, memory/heap, garbage collection, threads, databases, messaging systems, and network latency
• Experience creating reusable, maintainable, and portable automation and performance test scripts
• RAG testing experience, including embedding quality, retrieval evaluation, and chunk strategy validation
• Familiarity with vector databases and semantic search concepts
• Hands-on experience using AI assisted coding and analysis tools such as GitHub Copilot, Claude Code, or similar
• Ability to apply AI tools effectively for test automation development and refactoring, debugging and root cause investigation, exploratory test design and edge case discovery
• Strong understanding of limitations and risks of AI generated outputs, with the ability to validate, correct, and harden results for production quality use
• Experience with source control tools such as GitHub, Bitbucket, Git Bash
• Experience with test management tools (qTest, TestRail, ALM, TestLink, or similar)
• Familiarity with microservices and distributed system architectures
• Experience benchmarking, capacity planning, and release readiness reporting
• Knowledge of healthcare software, data privacy, and regulatory compliance is a plus
• Ability to manage multiple priorities and work independently in a fast paced environment