Develop and maintain data pipelines in Big Data environments (batch and, when necessary, near real-time);
Implement data ingestion, transformation and load processes (ETL/ELT);
Work on data modeling in distributed environments (Hive and Impala);
Manage and optimize queries over large volumes of data;
Orchestrate data workflows using Apache Airflow;
Develop scripts and applications in Python for data processing;
Use notebooks (Jupyter and Hue) for exploration, development and testing;
Ensure data quality, integrity and governance;
Implement best practices for versioning, documentation and observability;
Support the construction of the semantic layer for consumption by BI tools;
Develop APIs for data ingestion and extraction;
Create RESTful services for integration between internal and external systems;
Implement validation, transformation and data control routines in APIs;
Ensure performance and scalability when consuming large volumes of data;
Maintain and evolve existing APIs, ensuring stability and compatibility;
Implement access control and security (authentication and authorization);
Monitor and log executions for traceability and auditing;
Document APIs using Swagger/OpenAPI.

Bachelor’s degree completed (or equivalent);
Solid experience in data engineering within Big Data and Data Lake environments;
Advanced knowledge of Python for data processing;
Experience with distributed data modeling (Hive and Impala);
Experience orchestrating pipelines using Apache Airflow;
Experience building RESTful APIs and integrating systems;
Knowledge of data governance, quality and observability practices;
Experience with code versioning and technical documentation;
Knowledge of analytical layer architectures (RAW, TRT, RFD / Bronze, Silver, Gold);
Ability to optimize queries over large volumes of data;
Capacity to solve complex problems;
Emotional intelligence, teamwork, leadership and proactivity.

Company-subsidized Health Plan for the employee;
Option to include dependents in the Health Plan with payroll deduction;
Dental Assistance (optional);
Option to include dependents in the Dental Assistance with payroll deduction;
Meal Allowance or Food Voucher;
Transportation Voucher (optional);
Impact & Care
Personal Guidance Program offering emotional support and confidential counseling in psychological, legal, financial, social and pet-related matters at no cost for the employee and legal dependents;
Gympass
Wellhub (access to over 700 gyms across Brazil with plans starting at R$ 29.90 deducted from payroll);
Option to include dependents in Gympass
Wellhub (up to 3 dependents
paid via credit card);
Access to Udemy through our intranet;
Partnerships with major consumer brands;
Agreement with SESC for employee and dependents;
Discounts agreements with educational institutions (undergraduate and graduate) and language/certification schools;
Group life insurance.

Data Engineer – Senior

Key skills