About this role

Design and develop automated tests for Databricks data pipelines and transformations.
Validate end-to-end data movement across Bronze, Silver, and Gold data layers.
Create SQL and Python-based validation scripts to verify data quality and business rules.
Build and maintain automated regression test suites for data pipelines.
Integrate automated testing into CI/CD pipelines.
Implement data observability checks, including schema validation, data freshness, volume monitoring, and anomaly detection.
Investigate data quality issues, perform root cause analysis, and support production releases.
Collaborate with Data Platform Developers, Product Owners, and business stakeholders in an Agile environment to ensure high-quality data products.

Hands-on experience with Databricks (mandatory).
Experience testing data platforms, ETL/ELT pipelines, data lakes, or data warehouses.
Strong SQL skills for data validation, analysis, and troubleshooting.
Experience developing test automation using Python.
Experience with test automation frameworks and CI/CD pipelines.
Understanding of data quality, business rule validation, and data observability concepts.
Experience working in Agile teams alongside Data Platform Developers and Product Owners.
Strong analytical, problem-solving, and communication skills.

Data Platform QA Engineer, Data Quality, Observability

Key skills