Designing, building, and optimising ETL/ELT pipelines using Databricks (Spark/SQL), Delta Lake, and Databricks Workflows/Jobs
Developing and maintaining transformation models with dbt (Databricks adapter), including testing, documentation, and deployment via CI/CD
Implementing medallion architecture patterns (bronze/silver/gold) and best-practice data modelling to support analytics and self-service consumption
Building data quality checks and observability (e.g., dbt tests, expectations, monitoring/alerting) and supporting lineage and documentation
Applying security and governance using Unity Catalog (access controls, table ownership, auditability) and partnering with stakeholders on compliant data access
Performance tuning and cost optimisation (cluster policies, job orchestration, Spark optimisation, partitioning, Delta optimisation) and supporting incident/problem resolution
Requirements
A bachelor’s degree in computer science, information systems, or a related field
At least 3 years of relevant data engineering experience, including delivery on the Databricks Lakehouse Platform
Current Databricks certification(s) are required (e.g., Databricks Data Engineer Associate/Professional)
Hands-on experience with dbt (data build tool) for SQL-based transformations, testing, and documentation
Programming Skills – Python for data processing
Strong Spark and Databricks notebooks experience for data transformation and processing
Solid skills in SQL, experience with transformations, data warehouses and dimensional models
Experience with Delta Lake tables, optimisation and medallion architecture concepts
Experience with orchestration (Databricks Workflows/Jobs) and CI/CD for data pipelines (e.g., Azure DevOps/GitHub)
Understanding of governance and security patterns (preferably Unity Catalog)
Ability to work independently and as part of a team
Tech Stack
Azure
ETL
Python
Spark
SQL
Unity
Benefits
Employee benefits, recognition, and wellbeing platform
Career growth via succession planning, internal promotions, and mentorship opportunities
Ongoing investment in professional development through industry and technology certifications, and study assistance
Mental health support through our Employee Assistance Program
Support for family and caring responsibilities, including paid parental leave
Employee referral program, with monetary incentives offered