Key skills
AirflowApacheAWSAzureCloudGoogle Cloud PlatformJavaKafkaKubernetesNoSQLPythonScalaSparkSQLData EngineeringApache AirflowGCPGoogle Cloud
About this role
Role Overview
- Shape, develop, and maintain scalable real-time data pipelines and infrastructure
- Integrate various data sources and ensure seamless real-time data flow across the organisation
- Build efficient, fault-tolerant, and highly available data ingestion processes
- Monitor and improve data pipeline performance and scalability for low-latency and high-throughput
- Partner closely with cross-functional teams
- including software engineers, data scientists, and business stakeholders
- to ensure data infrastructure aligns with business objectives
Requirements
- Solid hands-on experience in data engineering, with a strong track record of building and managing real-time data pipelines across multiple initiatives
- Strong expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.)
- Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage
- Programming skills in Python, Java, Scala, or a similar language
- Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling
- Solid understanding of data pipeline orchestration tools (e.g. Apache Airflow, Kubernetes)
- Excellent written and verbal English skills.
Tech Stack
- Airflow
- Apache
- AWS
- Azure
- Cloud
- Google Cloud Platform
- Java
- Kafka
- Kubernetes
- NoSQL
- Python
- Scala
- Spark
- SQL
Benefits
- Private Health Care (Bupa)
- Hybrid Working (3 days in office)
- Travel Insurance
- Competitive Base Salary
- Company Bonus Scheme
- Market-Leading Training Programme
- Recognition & Reward Scheme
- Annual Company Conference (previous destinations: Vienna, Bologna, Dubrovnik, Thessaloniki)
- Regular Happy Hours & Team Lunches
- Free Coffee, Drinks & Snacks