Key skills
AirflowApacheAzureETLHadoopMongoDBOraclePythonSparkSQLELTData LakeDatabricksApache SparkApache AirflowSQL Server
About this role
Role Overview
- Build and maintain data ingestion and processing pipelines from multiple sources
- Develop extraction, transformation and load (ETL/ELT) processes using Python, Apache Airflow and Azure Data Factory
- Structure and manage data in the Data Lake, ensuring organization and data governance
- Implement transformations and data processing using Spark / SparkSQL on Databricks
- Support, monitor and evolve the client’s data platform, ensuring stability and performance
- Work with various databases and storage technologies, such as SQL Server, Oracle, MongoDB and Hadoop
- Collaborate with multidisciplinary teams in a dynamic, data-driven environment
Requirements
- Strong experience with Azure Databricks
- Knowledge of Azure Data Factory and Azure Data Lake
- Experience developing with Python
- Knowledge of Apache Spark / SparkSQL
- Experience with Oracle and SQL Server databases
Tech Stack
- Airflow
- Apache
- Azure
- ETL
- Hadoop
- MongoDB
- Oracle
- Python
- Spark
- SQL
Benefits
- Culture of equality
- Diversity and inclusion