Design, build, and maintain data ingestion and transformation pipelines using Databricks
Develop and support data pipelines across Bronze, Silver and Gold layers
Implement batch and streaming data pipelines using tools such as Databricks, Spark, and Azure Data Factory (ADF)
Work closely with Data Architects and business stakeholders to understand requirements
Support the implementation of data quality checks, validation rules, and monitoring processes
Assist in optimising data pipeline performance, scalability, and cost efficiency
Follow established best practices for data governance, lineage, and metadata management
Support testing, deployment, and release processes
Create and maintain technical documentation
Stay up to date with Databricks features and modern data engineering practices

Bachelor’s degree in IT, Computer Science, or a related field
3–6 years’ experience in data engineering, data warehousing, or related roles
Hands-on experience with Databricks and Spark-based data processing, including building and maintaining data pipelines
Experience working with ETL/ELT pipelines using tools such as Databricks, Azure Data Factory (ADF), or similar frameworks
Experience working with enterprise data stores such as SQL Server, PostgreSQL, Oracle, Synapse, or similar
Exposure to cloud-based data platforms (Azure preferred) and distributed data processing environments
Basic understanding of reporting tools such as Power BI or Tableau, and how data pipelines support analytics and dashboards
Experience within Financial Services or regulated environments is beneficial

Senior Databricks Data Engineer

Key skills