Key skills
AirflowApacheAWSKafkaPythonSparkSQLApache AirflowdbtLambdaCloudWatchKinesisDatadogMentoring
About this role
Role Overview
- Responsible for building core infrastructure software (pipelines, APIs, data modelling) as part of our client's data platform team
- Instrumenting systems for performance and enhancement throughout
- Ensuring data offerings to various internal & external stakeholders using secure authentication patterns
- Choosing and implementing appropriate technologies for scaling data access patterns, batch processing, and data streaming
- Coaching & mentoring other engineers to support technical expertise growth
Requirements
- Proven experience writing highly maintainable, performant Python code
- Experience building modern data pipelines using dbt, Kafka, Spark, AWS Kinesis, AWS Lambda, and Apache Airflow (or similar)
- Understanding of Data Modelling patterns
- Deep knowledge of complex SQL, with emphasis on Common Table Expressions, window functions, and their performance
- Experience with end-to-end monitoring & alerting experience (CloudWatch, Datadog, etc.)
Tech Stack
- Airflow
- Apache
- AWS
- Kafka
- Python
- Spark
- SQL
Benefits
- Flexible working
- Remote-first working environment
- Work-life balance comes as standard for all employees