Software Quality Assurance Engineer

Servamind is building a universal standard for AI infrastructure through its ServaStack data format and Chimera compute engine. The Software Quality Assurance Engineer will design and validate software quality processes across the SDLC, including automated, integration, performance, security, and benchmark testing for SaaS and on-premise deployments.

Responsibilities:

• Your primary responsibility, under the direction of our product leads, is to design, orchestrate, and validate the systems that gate every change shipped through the Software Development Lifecycle (SDLC) at scale
• You will own the quality assurance (QA) processes and results, including integration, end-to-end, regression, performance and security testing
• You will be key to ensuring that our products get into the hands of our users with few issues, ensuring happy, long-term customers
• You will also own benchmark testing
• Ensuring proper environmental and data variables are documented and understood
• Providing apples-to-apples testing
• You will interface with researchers and engineers to
• Ensure you understand the QA/testing requirements beyond what is documented
• Understand the functionality of the associated repos and identify edge cases to test
• Develop testing processes, including creative solutions to break products
• Ensure you have proper and up-to-date data for all tests
• Report deficiencies, including root cause analysis, found during testing
• You will test both SaaS (initial focus) and on-premise software deployments
• You will ensure that no critical Intellectual Property can be exposed in any deployment
• You will test scenarios of rolling back changes to ensure pre-change consistency and stability
• You will test to ensure compliance with SOC2 and related guidelines
• Ensure all API functionality is fully supported in the code
• Maintain clear documentation of how all tests are run
• Produce and distribute clear and concise test results, primarily in an automated manner
• Strive for the fastest feedback possible
• Automate as many tests as possible instituting more automation over time
• Develop unit and others tests, building libraries for reuse and automation
• Communicate clearly and regularly with team on potential updates, changes, or findings
• Provide professionally looking reports and/or documentation for key stakeholders, particularly shareholders
• Create and provide professional demos
• In some cases in collaboration with researchers and/or engineers

Requirements:

• Bachelor's degree in Computer Science, Software Engineering, or a related field
• 3 - 5 years experience with unit testing, integration testing, end-to-end testing, regression testing, performance testing and security testing
• Extensive experience in ML pipelines and testing LLMs
• Exceptional at identifying and testing edge cases
• Experienced Python developer with strong knowledge of Git and GitHub
• Proven experience comprehensively testing API services
• Experience creating and testing Continuous Improvement/Continuous Deployment pipelines
• Experience with DevOps testing and deployment
• Experience deploying and operating test automation on at least one major cloud platform (AWS, GCP, Azure)
• Experience automating workflows around software development and testing
• specific knowledge setting up and executing GitHub pipelines
• Strong familiarity with execution monitoring and debugging tools
• particularly for background processes and long-running processes
• tracing errors and inefficient code
• strong knowledge of tooling such as Grafana and Python debuggers
• Ability to learn new products, requirements and technologies quickly and effectively
• particularly, our technology is revolutionary in the ML space utilizing HDC
• Extensive experience in automation frameworks (e.g., pytest, Selenium) and scripting
• Experience with logging and debugging in distributed environments
• Excellent written and verbal communication skills, with the ability to articulate technical information to both technical and non-technical stakeholders
• Strong interpersonal skills with the ability to communicate effectively and interact appropriately with management, developers, etc
• Able to work independently in a remote, async environment
• AWS experience preferred
• Have created harnesses and infrastructure around AI models to detect model drift
• Familiarity with load and distribution testing across multiple GPUs at scale
• A foundation across deep learning types (i.e., computer vision, time series, generative models, reinforcement learning, and knowledge graphs)
• Experience testing multiple versions of software running simultaneously, properly load balancing capacity (ramping up volume to latest version) and measuring results (performance, output, etc.)
• Familiarity with SOC2 compliance requirements as well as related guidlines
• Knowledge of secure testing practices, OWASP guidelines, and SAST/DAST/SCA tooling
• Experience collaborating with engineers to define and implement mechanisms to inject testing earlier into the SDLC
• Comfortable with new and unique mathematics and its application