Data Engineer

Jobot is a leading technology-enabled platform dedicated to connecting talented professionals with innovative organizations across various industries. They are seeking a highly skilled Azure Databricks & Data Factory Specialist to design, develop, and maintain data solutions using Azure cloud services, focusing on building scalable data pipelines and ensuring data quality and security.

Responsibilities:

• Design, develop, and optimize data pipelines using Azure Data Factory and Databricks to support business analytics and reporting needs
• Create and maintain scalable data models and schemas within Data Lake and Lakehouse environments
• Implement data security, governance, and compliance best practices across all data solutions
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications
• Develop and automate data workflows to improve efficiency and reduce manual intervention
• Monitor and troubleshoot data pipelines, ensuring high availability and performance
• Maintain version control and documentation for all data engineering projects
• Stay updated on emerging trends and technologies in cloud data engineering to continuously improve our data infrastructure

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or a related field
• Proven experience with Azure Databricks, Azure Data Factory, and related Azure cloud services
• Strong proficiency in SQL, Python, and Scala for data processing and scripting
• Experience with Data Lake and Lakehouse architectures
• Knowledge of data modeling, data warehousing, and ETL/ELT processes
• Familiarity with CI/CD pipelines, Git version control, and DevOps practices
• Excellent problem-solving skills and ability to work collaboratively in a team environment
• Strong communication skills to effectively articulate technical concepts to non-technical stakeholders
• Master's degree