Data Engineer – Senior
Brazil
Full Time
2 hours ago
No H1B
Key skills
AzureETLKafkaPythonScalaSparkSQLAIELTData LakeAnalyticsBIPower BIDatabricksAzure DevOpsAzure SQLVersion ControlCI/CDCommunication
About this role
Role Overview
- Data Architecture: Design and implement scalable, efficient data architectures on Azure, ensuring data integration and data quality.
- ETL (Extract, Transform, Load): Develop and manage ETL pipelines to move and transform data between different systems and sources.
- Data Management: Monitor and optimize database performance, ensuring data security and integrity.
- Data Analysis: Collaborate with data scientists and analysts to provide clean, structured data for analysis.
- Process Automation: Automate repetitive tasks and data processes to improve operational efficiency.
- Data Governance: Implement data governance policies and procedures to ensure compliance with regulations and best practices.
- Support and Maintenance: Provide ongoing support and maintenance for data systems, troubleshooting issues and implementing improvements
Requirements
- Minimum of 5 years of experience as a Data Engineer
- Strong experience with Azure Data Services, including:
- Azure Synapse Analytics
- Azure Data Factory
- Azure Data Lake Storage
- Azure SQL Database
- Azure Databricks
- Experience developing with:
- SQL
- Python
- Scala (preferred)
- Experience with ETL and ELT tools and concepts
- Experience with relational and non-relational data modeling
- Experience with data architectures and Data Lake
- Experience with version control and development best practices
- Experience with data integration via APIs
- Experience in mission-critical enterprise environments
- Nice to have:
- Experience with Spark, Kafka or Big Data technologies
- Experience with Power BI or visualization tools
- Experience with CI/CD and Azure DevOps
- Experience with data governance and security
- Experience in large-scale enterprise environments
- Experience on data projects focused on analytics or AI
- Behavioral skills:
- Strong analytical and problem-solving skills
- Good communication with technical and business teams
- Organization and a strong sense of responsibility
- Ability to work autonomously
- Proactivity and a focus on quality
- Ability to operate in a dynamic, collaborative environment
Tech Stack
- Azure
- ETL
- Kafka
- Python
- Scala
- Spark
- SQL
Benefits
- Position also open to candidates with disabilities (PwD)