Key skills
AzureETLOraclePandasPySparkPythonSparkSQLELTAnalyticsBIDatabricksAzure DevOpsSQL ServerCI/CD
About this role
Role Overview
- Design, develop and maintain the backbone of our data platform
- Ensure data is robust, reliable and ready for analytical and strategic consumption
- Design, implement and maintain scalable pipelines, managing the full data lifecycle across Bronze, Silver and Gold layers on Databricks
- Build and manage ETL/ELT workflows using Azure Databricks (PySpark/SQL) and Azure Data Factory
- Ensure data integrity, consistency and performance with Delta Lake (ACID transactions)
- Collaborate with analysts and business teams to build dimensional models and aggregates optimized for BI and analytics
- Develop APIs and integrations for efficient data exchange between systems
- Work closely with BI, Data Science and Business teams, applying engineering best practices and data governance
Requirements
- Proven experience as a Data Engineer, with a focus on the Azure ecosystem
- Strong knowledge of Azure Databricks (Spark, PySpark and Spark SQL)
- Experience with Azure Data Factory (ADF) for pipeline orchestration
- Proficiency in SQL and Python (Pandas, PySpark)
- Solid knowledge of:
- ETL/ELT
- Medallion architecture
- Delta Lake
- Experience in data modeling (dimensional and relational)
- Experience with relational databases (Oracle, SQL Server)
- Differentials:
- Experience with dimensional modeling (Kimball methodology) in the Gold layer
- Knowledge of data governance and catalog tools (e.g., Azure Purview)
- Experience with CI/CD for data (e.g., Azure DevOps)
- Experience with data streaming (Spark Structured Streaming or Event Hubs)
- Relevant certifications (e.g., DP-203, Databricks)
Tech Stack
- Azure
- ETL
- Oracle
- Pandas
- PySpark
- Python
- Spark
- SQL
Benefits
- Health and dental insurance
- Meal and food allowance
- Childcare assistance
- Extended parental leave
- Partnerships with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass
- Profit Sharing (PLR)
- Life insurance
- Continuous learning platform (CI&T University)
- Discount club
- Free online platform dedicated to promoting physical and mental health and well-being
- Pregnancy and responsible parenthood course
- Partnerships with online course platforms
- Language learning platform
- And many more