Lead Software Engineer in Test – Agentic AI (Remote - US)

Nextech Systems is a leader in specialty healthcare technology solutions, seeking a Lead Software Engineer in Test for their Agentic AI team. This role involves leading quality engineering strategy and execution for AI-powered applications, focusing on the design of automation frameworks and ensuring high-quality delivery across various layers.

Responsibilities:

• Design, develop, and maintain scalable automation frameworks and regression test suites across UI, API, backend, and AI-driven applications
• Lead quality engineering initiatives for Agentic AI systems, including LLM workflows, RAG pipelines, autonomous agents, and distributed orchestration platforms
• Validate complex AI agent behaviors such as multi-agent coordination, tool calling, prompt chaining, memory retention, conversational workflows, and fallback/retry mechanisms
• Develop and execute functional, integration, regression, end-to-end, and performance testing strategies across cloud-native and distributed environments
• Collaborate with Software Engineers, Product Managers, and DevOps teams to integrate testing throughout the software development lifecycle
• Build reusable automation solutions to validate APIs, orchestration layers, asynchronous workflows, enterprise integrations, and AI-driven business processes
• Design and maintain robust test environments, CI/CD integrations, cloud execution platforms, and automation pipelines to improve efficiency and reliability
• Establish and drive best practices related to code quality, maintainability, scalability, dependency injection, observability, and testability
• Lead validation efforts for AI workflow resiliency, distributed system reliability, data consistency, and orchestration performance under failure and edge-case conditions
• Mentor QA engineers and contribute to the development of scalable QA standards, automation strategies, reporting practices, and continuous improvement initiatives

Requirements:

• Bachelor's degree in Computer Science, MIS, Engineering, or a related technical field
• 8+ years of experience in Software Development or Software Engineer in Test (SDET/SET) roles
• 5+ years of hands-on experience developing automation frameworks using JavaScript/TypeScript or Java
• Strong experience with automation tools such as Playwright, Cypress, Postman, K6, or equivalent frameworks
• 2+ years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows
• Hands-on experience validating agent orchestration workflows, conversational AI applications, multi-step reasoning systems, and AI-driven business processes
• Experience testing AI orchestration frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar technologies
• Strong expertise in API testing, backend integration testing, UI automation, and end-to-end workflow validation
• Solid understanding of automation architecture, design patterns, anti-patterns, dependency injection, and inversion of control principles
• Experience working in cloud-native and containerized environments (Azure, AWS, GCP, Docker, Kubernetes)
• Proven experience integrating automated test suites into CI/CD pipelines and distributed cloud environments
• Experience validating asynchronous, event-driven, and distributed systems with a focus on resiliency and fault tolerance
• Strong analytical, debugging, troubleshooting, and problem-solving skills
• Excellent organizational, communication, and stakeholder collaboration skills in Agile environments
• Experience with Azure DevOps, GitHub Actions, or similar CI/CD platforms
• Experience with AI observability tools, telemetry platforms, and prompt evaluation frameworks
• Experience validating RAG pipelines, vector databases, and AI response accuracy
• Knowledge of healthcare systems, HIPAA compliance, and regulated enterprise environments
• Experience with event-driven architectures, messaging systems, and distributed data platforms (including SQL Server)
• Prior experience leading QA teams, mentoring engineers, and implementing enterprise test reporting and quality metrics practices