Lead Data Platform Engineer (Databricks & Python)

New York Technology Partners is seeking a highly motivated Lead Data Platform Engineer to join their data team. In this pivotal role, you will architect and build scalable data solutions using Databricks and Python while collaborating with business stakeholders to translate requirements into technical solutions.

Responsibilities:

• Develop robust, scalable data pipelines and architectures using Databricks and Python
• Collaborate regularly with non-technical business partners to understand data needs and deliver actionable insights
• Write clean, efficient, and maintainable Python code; enforce best practices within the engineering team
• Act as a technical lead, mentoring junior engineers and driving self-started initiatives to improve platform performance
• Handle sensitive healthcare data with a focus on security, compliance, and integrity

Requirements:

• 12+ years of Data Engineering experience, with a focus on platform building
• Significant, hands-on experience with the Databricks Lakehouse Platform
• Superior programming ability in Python (not just scripting, but production-grade software development)
• Excellent verbal and written communication skills; able to explain technical concepts to business stakeholders
• Must be local to Minneapolis, MN (Required for this role)
• A curious self-starter who is motivated to solve complex problems without constant supervision
• Experience in the Healthcare industry (HIPAA compliance, HL7, FHIR, etc.)
• Previous experience in a Lead or Senior Individual Contributor role