Lead Data Engineer PennDnA- People

Penn Medicine is dedicated to providing the highest level of care to patients, conducting innovative research, and educating future leaders in the field of medicine. The Lead Data Engineer-People will be responsible for leading the design, development, and implementation of data engineering solutions to support data-driven initiatives.

Responsibilities:

• Lead the design and architecture of scalable and reliable data pipelines to ingest, process, and transform large volumes of healthcare data from diverse sources, including electronic health records (EHR), clinical systems, research databases, and external data feeds. Evaluate and recommend appropriate data technologies, platforms, and tools to optimize data processing, performance, and cost-effectiveness
• Coordinate and lead the build and optimization of ETL/ELT processes and work within Cloud technology such as AWS, Azure and/or Databricks to ensure efficient data extraction, transformation, and loading while maintaining data quality, consistency, and integrity. Collaborate with a variety of stakeholders across the organization, including senior leaders, to understand data requirements and translate them into effective ETL/ELT solutions that meet organizational goals
• Implement and maintain cloud-based data platforms and services, leveraging technologies such as Azure and Databricks. Design and deploy data solutions using cloud-native services to support advanced analytics, machine learning, and data visualization initiatives
• Guide the establishment and enforcement of data standards, data governance policies, and best practices to ensure the accuracy, completeness, consistency and integrity of healthcare data. Implement data monitoring, validation, and lineage tracking mechanisms to ensure compliance with regulatory requirements and organizational data governance policies, including definitional and quality standards
• Independently manage moderately complex projects from concept to execution, coordinating with other PennDnA teams as needed
• Build relationships with PennDnA and client stakeholders, including senior leaders. Collaborate with cross-functional teams to understand business needs and develop solutions that align with organizational goals. Help set the team’s direction, contributing to team strategy and roadmap
• Stay current with industry trends, best practices, and emerging technologies. Lead continuous improvement for data engineering. Contribute to a culture of performance excellence

Requirements:

• Bachelor of Arts or Science in Computer Science, Information Systems, or related field
• 6+ years Experience in data engineering, database management, or related roles, with significant experience in managing complex projects
• Proven track record of independently leading projects from concept to execution, coordinating with cross-functional teams, as needed
• Epic Clarity Data Model (must be completed within 6 months of start date)
• Master of Arts or Science