Modify or create new application resources in Kubernetes
Propose and implement architectural improvements
Implement LGPD-related controls in the data lake
Implement Databricks metadata dashboards and analyses for control and compliance
Conduct audits together with the governance team to align the data lake with the company's security policies
Monitor and analyze costs to identify bottlenecks and cost-reduction opportunities
Collaborate with data analytics and AI teams to optimize jobs and resource usage
Work with other platform engineers to support skill development
Serve as a technical point of reference for platform-related questions for data, business, and product stakeholders
Requirements
Bachelor's degree or higher
Solid knowledge of relational and non-relational databases: patterns, modeling and use cases
Experience with programming languages for maintaining data pipelines (Python, shell/bash) and file manipulation for this purpose (e.g., JSON, Parquet, CSV)
Experience consuming APIs and performing data cleaning, processing, deduplication, enrichment and normalization
Experience with messaging infrastructure using Kafka + Debezium
Experience with Kubernetes cluster infrastructure such as EKS and AKS
Strong SQL skills and basic knowledge of DBT
Experience with cloud computing projects (primarily AWS and/or Azure)
Experience with Gitflow and version control
Experience with Infrastructure as Code (Terraform)
Experience with distributed processing (Databricks/Spark)
Experience with Docker
Experience with orchestration tools such as Apache Airflow and Databricks Workflows
Knowledge of data architecture
Experience building CI/CD pipelines
Experience with observability tools (Grafana / CloudWatch)
Tech Stack
Airflow
Apache
AWS
Azure
Cloud
Docker
Grafana
Kafka
Kubernetes
Python
Spark
SQL
Terraform
Benefits
Meal allowance / meal card
Flexible working hours and remote arrangements (for remote positions)
Commuter allowance / transportation voucher (for hybrid or on-site positions)
Profit sharing (PLR)
Flexible benefits via Flash Card (Multibenefits)
Gympass / Wellhub
Psicologia Viva (online platform for therapy and nutrition consultations)
Health and dental insurance
Life insurance
Extended parental leave (up to 6 months for mothers and 20 days for fathers)
Rede D'Ór: support and important information for maternal and newborn health with a network of accredited nurses
Partnership with your regional SESC (varied programs in education, health, culture, leisure and social assistance)
Birthday Day Off (one paid day off on or during your birthday month)
Online learning platform with a variety of courses to enhance your skills (UCA)
Language academy (AIA)
Leadership development program
Mentoring for Afya Women (MMA)
Discounts on undergraduate and graduate courses at Afya's educational institutions