Senior Data Engineer

New York Technology Partners is collaborating with a digital consultancy seeking a Senior Data Engineer. The role involves designing, building, and maintaining scalable data solutions on Azure while working with both onshore and offshore teams.

Responsibilities:

• Collaborate with onshore and offshore engineering teams to deliver scalable data solutions
• Design and implement enterprise data models for analytics and reporting use cases
• Build and maintain data pipelines using Azure Databricks, PySpark, and Azure Data Factory
• Develop and optimize data warehouse solutions to support business intelligence and analytics
• Ensure data quality, reliability, and performance across pipelines and platforms
• Support data preparation and integration for AI/ML initiatives
• Participate in Agile/Scrum ceremonies and contribute to continuous improvement

Requirements:

• Strong focus in Azure Databricks
• Strong focus in Azure Data Factory
• Strong focus in Data Warehouses
• 5+ years of hands-on development experience with Python
• 6–8 years of experience writing complex SQL queries and optimizing performance
• 5+ years of experience with enterprise-level data modeling
• Strong experience with Azure Cloud services for data engineering
• Hands-on experience with Azure Databricks
• 5–7 years of experience building scalable data transformations with PySpark
• Experience creating and orchestrating pipelines using Azure Data Factory
• Healthcare experience
• Experience with Apache Airflow
• Exposure to or experience supporting AI/ML data pipelines
• Experience working in Agile/Scrum environments
• Prior experience working with globally distributed teams