Key skills
AzureCloudPySparkSparkSQLUnityData EngineeringDatabricksCloud Storage
About this role
Role Overview
- Lead end-to-end development of scalable data pipelines
- Design and implement Structured Streaming applications in Spark
- Collaborate on data governance and asset organization
- Manage Microsoft Fabric capacity and resource utilization
- Design and optimize Databricks dashboards and reporting artifacts
- Lead and mentor a team of data engineers
- Establish standards for data quality and governance
Requirements
- 10+ years of hands-on experience in data engineering on Azure
- Deep expertise in Azure Data Factory, Synapse, Databricks, and Microsoft Fabric
- Proven experience with real-time data processing and streaming architectures
- Strong proficiency in Spark (PySpark), SQL, and cloud storage
- Solid knowledge of Unity Catalog for data governance
- Experience with Databricks Dashboards
- Expertise in scalable, fault-tolerant data solutions
Tech Stack
- Azure
- Cloud
- PySpark
- Spark
- SQL
- Unity
Benefits
- Health insurance
- 401(k) matching
- Paid time off
- Flexible work arrangements
- Professional development
- Wellness programs