Work with a focus on data engineering, especially DataOps and MLOps, ensuring the data ecosystem is robust, efficient and reproducible for machine learning and generative AI projects;
Build and maintain data ingestion, monitoring and automation processes, ensuring reliability, performance and traceability of data across the platform;
Define standards, pipelines and technical solutions to accelerate the creation of new data products, promoting standardization and component reuse;
Implement and maintain tools and infrastructure on AWS (such as S3, Lambda, Redshift, GuardDuty), and be proficient in concepts and technologies such as data mesh, Python, Spark, SQL, DBT, Terraform, Kubernetes, Airflow and CI/CD;
Identify bottlenecks, map the needs of AI teams and propose evolution plans for the data platform, acting proactively and collaboratively with other specialists.
Requirements
Hands-on experience with AWS and its main services (S3, Lambda, Redshift, GuardDuty, DataZone, etc.);
Experience with data mesh architecture and implementing data pipelines;
Experience automating processes using Terraform and infrastructure-as-code tools;
Programming experience in Python, Spark and SQL, and experience using DBT for data transformation;
Experience with workflow orchestration (Airflow, Kubernetes) and CI/CD practices;
Knowledge of DataOps and MLOps, ensuring monitoring, performance and traceability of data;
Solid understanding of governance, security and data quality policies, including LGPD, PCI and EPIA;
Collaborative, curious and adaptable profile, comfortable working in dynamic and fast-changing environments.
Tech Stack
Airflow
Amazon Redshift
AWS
Kubernetes
Python
Spark
SQL
Terraform
Benefits
Profit Sharing
Company Car
Food Allowance
Meal Voucher
Health Insurance
Dental Insurance
Gympass
Private Pension Plan
Home Office Allowance
Allya
Unlimited access to a variety of courses through our Localiza University