Data Engineer - Databricks Certified - W2

SPECTRAFOR is seeking a Data Engineer with Databricks certification for a 7-month remote project. The role involves moving data from source systems to the Azure data lake and requires extensive experience with Databricks and Azure Data Factory, along with strong SQL skills.

Responsibilities:

• Data engineers that have extensive experience with Databricks as well as Azure data factory
• They must be able to unit test and have an understanding how to properly test the system ensure that the process is capturing all files / transactions which need to be processed
• They will be responsible for moving data from source systems to the Azure data lake for the business to use in reporting
• The engineers will need to know how to work with Unity Catalog and also know how to leverage delta tabels that are managed by databricks
• This person needs to be a Tech Lead person
• Pure data engineer: Databricks Certified, Spark programming, Azure Data Factory, Strong SQL Skills
• For 2 of the resources, they need to have PowerBI development experiance and able to help with performance testing

Requirements:

• Databricks Certified
• 5 years or more of Data Brick experience
• 5 years of Azure Data Factory experience
• 5 years of strong understanding of SQL
• Good verbal communication skills
• Understanding of Agile process
• Ability to unit test and understand how to properly test the system
• Experience with Unity Catalog
• Experience with delta tables managed by Databricks
• Ability to move data from source systems to the Azure data lake
• PowerBI development experience
• Ability to help with performance testing