You will gather and process raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks.
You will work on fast paced and innovative projects with the goal to deliver innovative solutions for our customers.
You will design, implement, and manage scalable data pipelines using cloud-native technologies to efficiently process and analyze large volumes of data.
You will evaluate and implement new technologies and tools in the area of data engineering to continuously improve the efficiency and performance of our data processing processes.
You will work closely with Data Scientists and Analysts, identifying potential data sources and evaluating their utility to understand data requirements and develop appropriate solutions.
Additionally you will share your expertise and experience with other team members, contribute to knowledge sharing and development of the team.
Finally you will develop and maintain documentation on data engineering processes, best practices and technologies.
Requirements
4+ years of hands-on experience in Data Engineering, working with large-scale data pipelines and cloud platforms.
Strong programming skills in Python and SQL, with experience using distributed data processing frameworks such as PySpark or Spark SQL.
Proficient in building and deploying cloud-native data pipelines in AWS.
Experience with AWS data and compute services, such as: S3, Glue, IAM, Redshift, Lambda, Step Functions, VPC, and DynamoDB.
Knowledge of CI/CD practices and Infrastructure-as-Code using Terraform, GitHub Actions, or similar tools.
Strong background in data integration, ETL/ELT development, data modeling, and data architecture.
Nice to have: Understanding of streaming technologies, such as Kafka, AWS Kinesis.
Nice to have: Experience working within Agile delivery environments.
Nice to have: Experience working in Unix/Linux environments, including writing Bash scripts for automation and operations.
Tech Stack
Amazon Redshift
AWS
Cloud
DynamoDB
ETL
Kafka
Linux
PySpark
Python
Spark
SQL
Terraform
Unix
Benefits
Professional & Personal Growth: Develop yourself both professionally and personally through training programs, free language courses, competence centers and an active tech community.
Flexible Work-Life Balance: Benefit from hybrid work, workation, flexible hours, parental support and sabbaticals.
Embrace Diversity & Sustainability: Engage in our Sustainability Hub, diverse communities, Diversity Taskforce and after-work activities.
Comprehensive Benefits: Enjoy public transport tickets, job bikes, health offers, supplementary insurances, a pension plan and various discounts.