Data Engineer

Jobot is a leading technology-enabled platform specializing in innovative staffing solutions and employment services. They are seeking a highly skilled Azure Databricks & Data Factory Specialist to design, develop, and maintain data pipelines and analytics solutions utilizing Azure cloud services.

Responsibilities:

• Design, develop, and optimize data pipelines using Azure Data Factory and Databricks to support enterprise data needs
• Implement and manage data lake and lakehouse architectures to facilitate scalable data storage and analytics
• Perform data modeling and schema design to ensure data integrity and accessibility
• Write efficient and maintainable code in Python and Scala for data processing tasks
• Collaborate with data analysts and business teams to understand data requirements and deliver actionable insights
• Maintain and improve CI/CD pipelines to ensure consistent deployment of data solutions
• Monitor and troubleshoot data workflows to ensure high availability and performance
• Ensure compliance with data security policies and best practices
• Stay updated with the latest developments in cloud data technologies and incorporate best practices into projects

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field
• Proven experience working with Azure Databricks and Azure Data Factory
• Strong knowledge of data lake and lakehouse architectures
• Expertise in data modeling, SQL, Python, and Scala programming languages
• Experience with CI/CD pipelines and version control systems such as Git
• Familiarity with data security, governance, and compliance standards
• Excellent problem-solving skills and the ability to work collaboratively in a team environment
• Strong communication skills with the ability to translate technical concepts to non-technical stakeholders