Azure Data Engineer – Databricks / PySpark / Synapse

eTeam is hiring an Azure Data Engineer to design, develop, and maintain scalable Azure-based data pipelines and platforms. The role focuses on Databricks, PySpark, SQL, ADLS, Synapse Analytics, data integration, performance optimization, governance, production support, and collaboration with stakeholders and data teams.

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark
• Extract, transform, and load (ETL/ELT) data from multiple source systems into Azure-based data platforms
• Develop and optimize large-scale data processing workflows using PySpark and Apache Spark
• Create, manage, and optimize database objects, including tables, views, stored procedures, and complex SQL queries
• Build and maintain data solutions using Azure Data Lake Storage (ADLS) and Azure Synapse Analytics
• Implement data integration and transformation processes to support reporting, analytics, and business intelligence requirements
• Monitor, troubleshoot, and improve the performance, reliability, and scalability of data pipelines
• Ensure data quality through validation, cleansing, reconciliation, and auditing processes
• Collaborate with business stakeholders, data analysts, and data scientists to understand requirements and deliver data solutions
• Implement data security, governance, and compliance standards across Azure data platforms
• Support production deployments, incident management, and ongoing operational support
• Create and maintain technical documentation, data flow diagrams, and operational procedures
• Participate in code reviews and follow best practices for data engineering and software development

Requirements:

• Strong experience with Azure Databricks, PySpark, and SQL
• Hands-on experience with Azure Data Lake Storage (ADLS) and Azure Synapse Analytics
• Strong understanding of ETL/ELT processes and data warehousing concepts
• Experience in performance tuning and optimization of Spark and SQL workloads
• Knowledge of data modeling techniques, including dimensional and relational modeling
• Familiarity with Git, Azure DevOps, CI/CD pipelines, and Agile methodologies
• Excellent analytical, troubleshooting, and problem-solving skills