Develop and optimize high-performance, scalable batch data pipelines for ingesting structured, semi-structured, and unstructured data into our Data Lake (GCS);
Develop and manage ETL/ELT jobs for data processing using Spark/SQL to populate the Trusted (Silver) and Refined (Gold) layers of our Data Warehouse (BigQuery);
Consolidate and model data from internal and external sources to enable Analytics, Business Intelligence, Data Science, GenAI initiatives, and Data Products;
Develop online data integrations (streaming) for data ingestion and processing on GCP;
Design and implement efficient data architectures, optimizing development time, performance, and compute cost;
Reproduce business rules within data transformation processes to generate datasets;
Create and maintain Data Quality, Data Assurance, Lineage, and Data Versioning processes;
Serve as the Technical Reference for Matrix Energia's Data & Analytics team, supporting the most innovative and ambitious initiatives in our day-to-day work.
Requirements
Bachelor's degree in Computer Science, Systems, Technology, Data, or a related field;
Strong experience with Python and Spark/SQL;
Experience with CI/CD, preferably using GitHub;
Experience with Google Cloud Platform (GCP), Google Cloud Storage (GCS), and Google BigQuery;
Experience with data orchestration tools such as Airflow or Prefect;
Experience with data streaming technologies such as Pub/Sub or Azure Functions;
Knowledge of Data Modeling using the Medallion methodology.
Tech Stack
Airflow
Azure
BigQuery
Cloud
ETL
Google Cloud Platform
Python
Spark
SQL
Benefits
High-quality health coverage with Omint: access to a top-tier medical and dental network with nationwide coverage.
Alelo Pod card for flexible meal and grocery expenses, accepted at major restaurant chains and supermarkets.
Flexible and sustainable working hours.
Mobility: choose the option that works best for you — transit voucher, fuel allowance, or parking.
Total Pass: an online platform dedicated to promoting quality of life, health, and well-being. Plans provide access to gyms, studios, and online services with exclusive Matrix discounts, covering you and up to 3 dependents.
Group life insurance.
Profit Sharing (PLR), subject to target achievement and performance.
Work with a fantastic team in a collaborative environment focused on building a brighter, more sustainable future.