Key skills
AzureETLPythonSparkSQLUnityELTData EngineeringAnalyticsDatabricksdbtAzure DevOpsGitHubCI/CD
About this role
Role Overview
- Designing, building, and optimising ETL/ELT pipelines using Databricks (Spark/SQL), Delta Lake, and Databricks Workflows/Jobs
- Developing and maintaining transformation models with dbt (Databricks adapter), including testing, documentation, and deployment via CI/CD
- Implementing medallion architecture patterns (bronze/silver/gold) and best-practice data modelling to support analytics and self-service consumption
- Building data quality checks and observability (e.g., dbt tests, expectations, monitoring/alerting) and supporting lineage and documentation
- Applying security and governance using Unity Catalog (access controls, table ownership, auditability) and partnering with stakeholders on compliant data access
- Performance tuning and cost optimisation (cluster policies, job orchestration, Spark optimisation, partitioning, Delta optimisation) and supporting incident/problem resolution
Requirements
- A bachelor’s degree in computer science, information systems, or a related field
- At least 3 years of relevant data engineering experience, including delivery on the Databricks Lakehouse Platform
- Current Databricks certification(s) are required (e.g., Databricks Data Engineer Associate/Professional)
- Hands-on experience with dbt (data build tool) for SQL-based transformations, testing, and documentation
- Programming Skills – Python for data processing
- Strong Spark and Databricks notebooks experience for data transformation and processing
- Solid skills in SQL, experience with transformations, data warehouses and dimensional models
- Experience with Delta Lake tables, optimisation and medallion architecture concepts
- Experience with orchestration (Databricks Workflows/Jobs) and CI/CD for data pipelines (e.g., Azure DevOps/GitHub)
- Understanding of governance and security patterns (preferably Unity Catalog)
- Ability to work independently and as part of a team
Tech Stack
- Azure
- ETL
- Python
- Spark
- SQL
- Unity
Benefits
- Employee benefits, recognition, and wellbeing platform
- Career growth via succession planning, internal promotions, and mentorship opportunities
- Ongoing investment in professional development through industry and technology certifications, and study assistance
- Mental health support through our Employee Assistance Program
- Support for family and caring responsibilities, including paid parental leave
- Employee referral program, with monetary incentives offered