Build and maintain distributed data pipelines using Scala, Spark, and cloud technologies
Collaborate with engineers, data scientists, and product teams to deliver reliable, scalable data systems
Design and optimise data ingestion and transformation workflows across blockchain and traditional datasets
Ensure accuracy, scalability, and efficiency in systems processing hundreds of millions of daily data points
Evaluate design options and trade‐offs across performance, scalability, reliability, and cost
Contribute to the full lifecycle of data platform development from design and deployment to continuous improvement
Strengthen pipeline reliability, observability, and automation through code and tooling improvements
Grow your influence and take on greater responsibility as you deepen your understanding of our distributed systems and platform architecture
Requirements
Experience delivering and maintaining distributed data pipelines
Practical knowledge of Spark, Databricks, or similar big data frameworks
Familiarity with cloud infrastructure (AWS, Azure, or GCP)
Understanding of data-architecture trade‐offs, such as scalability, resilience, and observability
Interest or experience in functional programming
Bonus points for: Experience in streaming processing concepts like delivery semantics, ordering or partitioning
Hands‐on work with Infrastructure as Code (Terraform or CloudFormation)
Knowledge of container orchestration (Docker, Kubernetes, Helm)
An interest in blockchain and cryptocurrency technology, or a desire to learn
Experience applying AI or automation within deployed services
Tech Stack
AWS
Azure
Cloud
Distributed Systems
Docker
Google Cloud Platform
Kubernetes
Scala
Spark
Terraform
Benefits
Hybrid working and the option to work from almost anywhere for up to 90 days per year
£500 Remote working budget to set up your home office space
$1,000 Learning & Development budget to use on anything (agreed with your manager) that contributes to your growth and development
Holidays: 25 days of annual leave + bank holidays
An extra day for your birthday
Enhanced parental leave: we provide eligible employees, regardless of gender or whether they become a parent by birth or adoption, 16 weeks of fully paid leave.
Private Health Insurance
we use Vitality!
Full access to Spill Mental Health Support
Life Assurance: we hope you will never need this
but our cover is for 4 times your salary to your beneficiaries