Key skills
Amazon RedshiftApacheAWSETLPythonSQLAIMachine LearningMLRAGELTData EngineeringAnalyticsRedshiftLambdaS3GlueCI/CDRemote Work
About this role
Role Overview
- Design, develop, and maintain automated QA frameworks for data pipelines, APIs, and analytics platforms using Python and SQL.
- Build reusable testing utilities for data validation, regression testing, and pipeline certification.
- Integrate automated tests into CI/CD pipelines to support continuous testing and deployment.
- Develop unit, integration, and end-to-end test cases for complex data workflows.
- Leverage AI-assisted testing tools to generate test cases, identify edge cases, and improve test coverage.
- Validate ETL/ELT pipelines to ensure accurate ingestion, transformation, and delivery of data.
- Create automated checks for data completeness, consistency, accuracy, and timeliness.
- Test ingestion and transformation of complex datasets, including XBRL financial data.
- Implement reconciliation and audit mechanisms across source-to-target mappings.
- Apply AI-driven anomaly detection to identify data quality issues and pipeline failures.
- Develop and execute test strategies for Apache Iceberg-based data lakehouse architectures.
- Ensure alignment between precomputed datasets (materialized views) and underlying source data.
- Implement automated validation for data quality rules, lineage, and metadata accuracy.
- Collaborate with data and AI teams to test data pipelines supporting RAG, analytics, and machine learning workflows.
- Document test results, defects, and quality metrics for both technical and non-technical stakeholders.
- Promote adoption of AI-driven efficiencies and automation across QA and data engineering workflows.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field.
- 5+ years of experience in QA engineering, data testing, or software development.
- Strong programming skills in Python and advanced proficiency in SQL.
- Experience building automated test frameworks for data platforms and ETL pipelines.
- Hands-on experience with: AWS data services (S3, Glue, Redshift, Lambda, etc.)
- Experience validating materialized views and performance-optimized data structures.
- Familiarity with XBRL or complex financial/regulatory datasets.
- Understanding of data modeling, metadata, and data governance principles.
- Experience with CI/CD tools and automated testing integration.
- Demonstrated proficiency with AI tools and AI-assisted development/testing workflows.
- Understanding of data quality requirements for AI/ML and analytics use cases.
- U.S. Citizenship required; ability to obtain and maintain a federal clearance.
Tech Stack
- Amazon Redshift
- Apache
- AWS
- ETL
- Python
- SQL
Benefits
- 100% remote work
- Professional development opportunities