Senior Azure Databricks Engineer

Diverse Lynx is seeking a Data Engineer with strong hands-on experience in Azure Cloud and related technologies. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and cloud-based data solutions.

Responsibilities:

• Designing, developing, and optimizing scalable data pipelines and cloud-based data solutions
• Expertise in ETL/ELT processing, data modeling, performance tuning, and data governance
• Proficiency in Python, SQL, GitHub, and CI/CD practices
• Working with large-scale datasets in enterprise environments
• Demands strong analytical and problem-solving skills
• Ability to work independently
• Ownership of end-to-end solution delivery
• Collaborating effectively with cross-functional teams and business stakeholders

Requirements:

• strong hands-on experience in Azure Cloud
• strong hands-on experience in Azure Databricks
• strong hands-on experience in PySpark
• strong hands-on experience in Azure Data Factory (ADF)
• strong hands-on experience in Delta Lake
• strong hands-on experience in Unity Catalog
• expertise in ETL/ELT processing
• expertise in data modeling
• expertise in performance tuning
• expertise in data governance
• proficiency in Python
• proficiency in SQL
• experience with GitHub
• experience with CI/CD practices
• experience working with large-scale datasets in enterprise environments
• strong analytical and problem-solving skills
• ability to work independently
• ownership of end-to-end solution delivery
• ability to collaborate effectively with cross-functional teams and business stakeholders