Key skills
AirflowBigQueryCloudDockerGoogle Cloud PlatformKafkaPySparkPythonSparkSQLTerraformData EngineeringAnalyticsGCPGoogle CloudPub/SubGitHubBitbucket
About this role
Role Overview
- Develop and maintain data ingestion and transformation pipelines.
- Integrate data from multiple sources.
- Build analytical datasets in the Data Warehouse.
- Ensure data quality, performance, and governance.
- Monitor pipelines and implement quality controls.
- Model data for analytics.
- Support the definition of metrics and KPIs.
- Document processes and pipelines.
- Participate in deployment and versioning of code.
- Collaborate with business and technology teams.
Requirements
- Python for data engineering.
- Advanced SQL.
- Airflow.
- BigQuery.
- GCP (Google Cloud Platform).
- (Bitbucket/GitHub) and best practices for versioning data pipelines.
Nice to have:
- Spark / PySpark.
- Docker.
- Terraform.
- Streaming (Kafka / Pub/Sub).
- Dimensional modeling.
Tech Stack
- Airflow
- BigQuery
- Cloud
- Docker
- Google Cloud Platform
- Kafka
- PySpark
- Python
- Spark
- SQL
- Terraform
Benefits
- Meal allowance or food voucher to buy groceries or dine at your preferred restaurant;
- Health and dental insurance for you and your family;
- Private pension plan for your financial security and investments;
- Childcare assistance to support you during your children's early years;
- Life insurance to provide peace of mind for you and your loved ones;
- Profit-sharing as an incentive for your work and results;
- Well-being programs that promote employees' health and quality of life;
- Educational grant programs to support your professional development;
- Gympass, giving you access to over 18,000 gyms across Brazil.