Key skills
AzureERPETLPySparkScalaSparkSQLTerraformUnityELTData EngineeringAnalyticsBIPower BIDatabricksCRMCI/CDCommunication
About this role
Role Overview
- Build and maintain ETL/ELT pipelines in Azure Data Factory, integrating sources such as ERP/CRM/APIs/streaming into a Lakehouse (Delta Lake) and consumption layers.
- Develop transformations and jobs in Databricks using Spark (PySpark/Scala) and SQL, ensuring performance, cost-efficiency, and reliability.
- Perform data modeling (bronze/silver/gold) and provide datasets for consumption in Power BI and/or the DW/Lakehouse.
- Act as a bridge between data engineering and BI, translating business requirements into analytical solutions.
- Implement governance best practices (catalog, ownership, standards, access controls) and support compliance with LGPD.
- Define and operate Data Quality, observability, and lineage with monitoring, alerts, and SLAs.
- Automate infrastructure and deployments using Terraform and CI/CD.
- Document solutions, design architecture, and participate in technical decisions and estimates.
Requirements
- Senior experience with Azure, including Databricks and Azure Data Factory.
- Strong knowledge of Delta Lake/Lakehouse, Spark (PySpark/Scala), and SQL.
- Experience with data modeling for analytics/BI and integration with Power BI.
- Experience with data governance/catalog and access controls (e.g., Unity Catalog and/or an enterprise catalog tool).
- Experience with Data Quality, observability, lineage, and practices aligned with LGPD.
- Experience with Terraform and CI/CD pipelines.
- Soft skills: business communication, autonomy, and priority management.
- Advanced English (meetings and documentation) — will be tested in the interview.
Tech Stack
- Azure
- ERP
- ETL
- PySpark
- Scala
- Spark
- SQL
- Terraform
- Unity
Benefits
- Market-competitive salary;
- Caju (flexible benefits card) with monthly top-up;
- Bradesco Health Plan;
- Bradesco Dental Plan;
- Preventive care with Dr. Alper;
- Life insurance;
- Gympass;
- SESC benefits;
- Childcare assistance for mothers and fathers;
- Learning: a department focused on developing hard and soft skills;
- Partnerships with educational institutions for technical training, MBA, postgrad programs, certifications, English and Spanish courses;
- Career development plan;
- Discounts on products from a partner portal