Data Engineer - Remote role

Apolis is seeking a Data Engineer for a 10-week Microsoft Fabric modernization and migration project. The role will transform SQL stored procedures into Fabric notebooks, build orchestration and validation frameworks, create silver-layer Lakehouse tables, and document the technical implementation.

Responsibilities:

• Take existing SQL stored procedures, transforming to fabric notebooks and those will create the silver tables in the Lakehouse in fabric, and they should match 1 to 1 SQL tables on PREM
• Creating orchestration framework using data pipelines in fabric
• Must do validate artifacts - reconciliation results, expectance evidence, making sure the tables match 1 to 1
• Build out validation tests that can consistently run day in and day out
• Will be building silver layer in the medallion architecture (will be in a solely isolated environment) - bronze is outside of scope and gold is beyond their score
• Creating technical documentation to cover what they have done -this is what we did and how

Requirements:

• 5-7 years of experience (SQL, T-SQL, Spark)
• Fabric (1-2 years minimum)
• Very strong SQL and T-SQL (understand stored procedures, understand SQL, taking that SQL and migrating it to fabric notebooks)
• PySpark
• Spark SQL
• Understand data flows in Fabric, data pipelines, understand what a Lakehouse is an how to use it
• What a medallion architecture will be primarily in the silver layer
• Some understanding the orchestration and scheduling in fabric
• Nice to have is experience using metadata framework