Senior Data Engineer - Synapse and Fabric

Bayforce is seeking a Senior Data Engineer to migrate enterprise data workloads from Azure Synapse Analytics to Microsoft Fabric. The role involves hands-on execution of migration tasks, collaborating with Data Architects and Engineers, and ensuring the implementation of Fabric-native solutions.

Responsibilities:

• Individually execute hands-on migration of enterprise data workloads from Azure Synapse Analytics to Microsoft Fabric, including pipelines, Spark notebooks, and SQL workloads
• Collaborate closely with Data Architects and fellow Data Engineers to translate target-state architecture into working Fabric solutions and ensure consistent implementation across migration efforts
• Review existing Synapse implementations (ADF/Synapse pipelines, Spark pools, SQL pools) and design, refactor, and implement Fabric-native equivalents using Lakehouse and Warehouse architectures (Bronze/Silver/Gold)
• Migrate data storage from ADLS Gen2 to OneLake, ensuring data parity, performance, security, and governance controls
• Build, optimize, and maintain end-to-end ELT/ETL pipelines using Microsoft Fabric Pipelines, Fabric notebooks, PySpark, SQL, and Azure Data Factory where required
• Develop and tune PySpark/Spark transformations for large-scale data processing, including incremental loads, partitioning strategies, joins, and file optimization
• Implement and support API-heavy ingestion frameworks (REST/SOAP), handling authentication, pagination, throttling, retries, and robust error handling
• Troubleshoot and resolve migration defects, performance regressions, and functional gaps between Synapse and Fabric
• Implement monitoring, logging, alerting, and operational runbooks for both migrated and net-new Fabric workloads
• Apply security, governance, and data quality standards across pipelines, including access controls, validation checks, lineage, and compliance requirements

Requirements:

• Bachelor's degree (or equivalent experience) in Computer Science, Engineering, or a related field
• 5+ years of hands-on data engineering experience in Azure cloud environments
• Strong, practical experience with Azure Synapse Analytics, including: Synapse Pipelines and/or Azure Data Factory, Synapse Spark pools, Dedicated and/or Serverless SQL pools
• Direct, hands-on experience migrating or modernizing data platforms, preferably from Synapse to Microsoft Fabric
• Hands-on Microsoft Fabric experience is REQUIRED, including: Fabric Lakehouse and Warehouse, OneLake, Fabric Pipelines, Fabric Notebooks, Dataflows Gen2
• Demonstrated experience rebuilding Azure Synapse patterns into Fabric-native solutions (not lift-and-shift)
• Advanced PySpark / Spark experience for complex transformations and performance tuning
• Strong SQL expertise and data warehousing concepts (dimensional modeling, incremental processing, data quality checks)
• Deep understanding of Data Lake, Lakehouse, and Warehouse architectures
• Proven ability to troubleshoot and optimize complex production data workflows