Principal Data Platform Engineer (Databricks)

Prominence Advisors is a healthcare and life sciences technology and consulting firm dedicated to helping organizations leverage their data. They are seeking a Principal Data Platform Engineer to lead the architecture and delivery of modern data platforms, focusing on client engagement and mentoring team members.

Responsibilities:

• Own the end-to-end architecture and delivery of scalable data solutions, with a strong emphasis on Databricks-based platforms and modern cloud ecosystems
• Lead design and implementation of data pipelines, data models, and transformation frameworks that support analytics, reporting, and advanced use cases
• Serve as a primary client-facing technical leader, building trusted relationships and guiding stakeholders through complex data challenges and solution decisions
• Translate ambiguous business requirements into clear, actionable technical architectures and delivery plans
• Establish and enforce best practices across data engineering, including g data ingestion (pipeline orchestration, testing, and optimization) and DevOps
• Drive platform strategy and architecture decisions, including lakehouse design, medallion architecture, and governance frameworks
• Lead and mentor delivery teams, providing technical guidance, code reviews, and hands-on support to ensure successful project outcomes
• Collaborate with cross-functional teams including data architects, analysts, and client stakeholders to ensure alignment and value delivery
• Identify risks, proactively address challenges, and ensure high-quality, timely delivery across engagements
• Contribute to internal capability building, including reusable frameworks, accelerators, and thought leadership
• Support business development efforts by shaping technical solutions, contributing to proposals, and participating in client discussions

Requirements:

• 7+ years of experience in data engineering, with demonstrated progression into technical leadership and architecture ownership
• Deep expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing
• Advanced proficiency in SQL and Python, with strong experience building and optimizing large-scale data pipelines
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including data services and infrastructure design
• Strong understanding of data modeling concepts, ETL/ELT patterns, and distributed data processing
• Experience with orchestration tools (e.g., Airflow) and transformation frameworks (e.g., dbt)
• Proven ability to lead technical delivery while remaining hands-on when needed
• Strong client-facing experience, including requirements gathering, solution design, and executive communication
• Ability to navigate ambiguity, prioritize effectively, and drive clarity in complex environments
• Healthcare data experience (e.g., Epic, HL7, FHIR, claims data) is strongly preferred
• Experience with CI/CD, DevOps practices, and infrastructure-as-code tools is a plus