Principal Data Engineer

Role Overview

• You’ll lead complex engineering work across critical data pipelines and platform capabilities, acting as the technical quality bar for data engineering across the Tribe.
• This will include:
• Leading and implementing complex engineering work across critical data pipelines and platform capabilities.
• Taking the lead on difficult production incidents, performance bottlenecks, and data integrity issues.
• Driving root-cause analysis and remediation for systemic engineering and operational weaknesses.
• Designing, building, optimising, and troubleshooting scalable data pipelines across batch and streaming workloads where applicable.
• Contributing production-grade Python and SQL code aligned to engineering standards for maintainability, readability, observability, and operational resilience.
• Leading optimisation and stabilisation efforts across Pentaho ETL and Databricks lakehouse ecosystems.
• Improving Spark workload efficiency, cluster utilisation, query performance, storage optimisation, orchestration reliability, and platform scalability.
• Defining and implementing observability standards for critical pipelines and datasets.
• Improving operational readiness across the data estate through monitoring, alerting, dashboards, runbooks, ownership clarity, and operational tooling.
• Improving developer experience through reusable frameworks, CI/CD patterns, templates, libraries, and “golden path” implementation examples.
• Promoting pragmatic and safe adoption of AI-assisted and agentic engineering approaches where they improve engineering outcomes.
• Supporting data-heavy and AI-enabled product initiatives including recommendation engines, behavioural scoring, propensity modelling, and intelligent automation features.
• Designing and building production services, APIs, and integration layers that expose new data capabilities and features into products and operational workflows.
• Developing squad engineering leads through mentorship, coaching, technical guidance, and hands-on support.
• Collaborating with Data Engineers, Platform Engineering, SRE, Architecture, Governance, Product teams and engineering leadership to unblock delivery and support complex engineering challenges.

Requirements

• Strong hands-on experience designing, building, operating, and optimising enterprise-scale data platforms and pipelines.
• Experience with Pentaho ETL or similar technologies in large-scale production environments.
• Strong hands-on experience with Databricks, Spark, Delta Lake, and modern lakehouse engineering patterns.
• Advanced Python and SQL skills, supported by solid software engineering fundamentals including testing, CI/CD, observability, packaging, and code quality.
• Experience designing and building production services, APIs, and integration layers that bring data capabilities into products and business workflows.
• A strong understanding of distributed data processing, scalability, resilience, and failure handling.
• Experience diagnosing and resolving complex production issues, including performance, instability, and data integrity problems.
• Good knowledge of modern data engineering patterns including CDC, idempotency, streaming concepts, backfills, partitioning, and workload optimisation.
• Experience operating production-grade data services with strong observability and operational excellence practices.
• The ability to influence engineering direction and raise capability across multiple squads through technical credibility and collaboration.
• Experience mentoring senior engineers or technical leads in a complex engineering environment.
• Strong communication skills, with the ability to explain complex technical topics clearly to a range of audiences.

It would also be great if you have:

• Experience operating hybrid legacy and modern data platform estates.
• Experience with cloud-based data ecosystems, Azure preferred.
• Experience supporting data and AI product development initiatives.
• Exposure to recommendation systems, behavioural analytics, propensity modelling, or intelligent automation platforms.
• Practical experience applying AI-assisted or agentic tooling within engineering workflows.
• Experience operating within highly regulated, high-availability, or operationally sensitive environments.

Tech Stack

• Azure
• Cloud
• ETL
• Python
• Spark
• SQL

Benefits

• A friendly, flexible and trust-based approach to working
• 25 days annual leave, plus usually a generous Christmas break
• Premium private healthcare and dental care with Uniqa, available from day one
• Flexi-Funds – a monthly allowance of 65 EUR to support bills and essential expenses in a way that works for you
• Food vouchers – 60 EUR per month
• Innovation and learning – space to develop skills, try new ideas and experiment, with an annual hackathon where some ideas make it into real work
• A great office setup – free snacks and drinks every day
• Access to the Storytel app for audiobooks and e-books
• Bede Bucks – exclusive colleague discounts and access to a wellbeing platform
• Lots of social events – both in and outside of working hours
• Referral programme – help us grow the team and receive a referral bonus of up to 3,000 EUR (pre-tax, subject to scheme terms)
• Bede swag – including hoodies, t-shirts and our much-loved Bede socks
• Bede Holidays – extra discretionary days off through the year as a thank-you for the great work our teams do