Data Engineer
Natal, Rio Grande do Norte, Brazil
Full Time
2 hours ago
No Sponsorship
Key skills
AirflowAWSAzureCloudDockerETLGoogle Cloud PlatformPandasPythonSparkSQLELTGCPGoogle CloudGitVersion ControlSales
About this role
Role Overview
- Develop data pipelines (ETL/ELT);
- Extract data from multiple sources (APIs, databases, files, etc.);
- Clean, transform, and standardize data;
- Model data for analytical consumption;
- Create and maintain DAGs in Airflow;
- Automate data workflows and schedules;
- Monitor pipeline failures and performance;
- Develop in Python (mainly using pandas and Spark);
- Use containers with Docker;
- Version control with Git;
- Integrate with cloud environments (AWS, GCP, or Azure);
- Translate business needs into data solutions;
- Support areas such as finance, operations, and sales.
Requirements
- Advanced knowledge of Python;
- Airflow;
- Docker;
- Knowledge of SQL;
- Experience with Pandas and Spark;
Tech Stack
- Airflow
- AWS
- Azure
- Cloud
- Docker
- ETL
- Google Cloud Platform
- Pandas
- Python
- Spark
- SQL