Azure Data Engineer

Programmers.io is currently looking for an Azure Data Engineer. The role involves designing and developing scalable data pipelines, orchestrating data workflows, and ensuring the performance and reliability of data solutions in the Azure Cloud environment.

Responsibilities:

• Design and develop scalable data pipelines using Scala Spark and Databricks in Azure Cloud
• Orchestrate end-to-end data workflows using Azure Data Factory
• Transform, parse, and load structured/unstructured data into Snowflake for downstream analytics
• Manage and optimize Spark clusters and performance tuning
• Develop Python scripts for data processing and automation
• Support API development and customization using Java Spring Boot
• Ensure high performance, reliability, and scalability of data solutions

Requirements:

• Design and develop scalable data pipelines using Scala Spark and Databricks in Azure Cloud
• Orchestrate end-to-end data workflows using Azure Data Factory
• Transform, parse, and load structured/unstructured data into Snowflake for downstream analytics
• Manage and optimize Spark clusters and performance tuning
• Develop Python scripts for data processing and automation
• Support API development and customization using Java Spring Boot
• Ensure high performance, reliability, and scalability of data solutions
• Python
• ADF
• Azure Functions
• Scala
• Spark
• Data Bricks
• Snowflakes
• SQL
• Strong understanding of data transformation patterns and cloud-based architecture best practices
• Experience working in enterprise-level data migration and analytics environments