Senior Data Engineer – Streaming, Analytics & Visualization

Highmark Health, through enGen and HM Health Solutions Inc., provides healthcare technology and data solutions. The Senior Data Engineer will design and operate real-time and analytical data pipelines using Azure Data Explorer, Kafka, Cribl, and Azure Data Factory, while developing curated datasets, analytics, and Power BI or Tableau dashboards. The role also supports data governance, security, stakeholder collaboration, and mentoring of other data engineers.

Responsibilities:

• Design, develop, and maintain robust data processes and solutions to ensure the efficient movement and transformation of data across multiple systems
• Develop and maintain data models, databases, and data warehouses to support business intelligence and analytics needs
• Collaborate with stakeholders across IT, product, analytics, and business teams to gather requirements and provide data solutions that meet organizational needs
• Monitor work against production schedule, provide progress updates, and report any issues or technical difficulties to lead developers regularly
• Implement and manage data governance practices, ensuring data quality, integrity, and compliance with relevant regulations
• Collaborate on the design and implementation of data security measures, including access controls, encryption, and data masking
• Mentor other associate and intermediate data engineers as needed
• Perform data analysis and provide insights to support decision-making across various departments
• Stay current with industry trends and emerging technologies in data engineering, recommending new tools and best practices as needed
• Other duties as assigned or requested

Requirements:

• CANDIDATE MUST BE US Citizen (due to contractual/access requirements)
• • 5 years of experience in design and analysis of algorithms, data structures, and design patterns in the building and deploying of scalable, highly available systems
• • 5 years of experience in a data engineering, ETL development, or data management role
• • 5 years of experience in SQL and experience with database technologies (e.g., MySQL, PostgreSQL, MongoDB)
• • 5 years of experience in data warehousing concepts and experience with data warehouse solutions (e.g., Snowflake, Redshift, BigQuery)
• • Demonstrated ability to achieve stretch goals in a highly innovative and fast-paced environment
• • Adaptability: Strong ability to take on diverse tasks and projects, adapting to the evolving needs of the organization
• • Analytical Thinking: Strong analytical skills with a focus on detail and accuracy
• • Interest and ability to learn other data development technologies/languages as needed
• • Technical Proficiency: Comfortable with a range of data tools and technologies, with a willingness to learn new skills as needed
• • Strong track record in designing and implementing large-scale data sources
• • Strong sense of ownership, urgency, and drive
• • Demonstrated passion for user experience and improving usability
• • Team Collaboration: A team player who can work effectively in cross-functional environments
• • Experience and willingness to mentor junior data engineers and help develop their skills and leadership
• • Bachelor's degree in Computer Science, Information Systems, Data Science, Computer Engineering or related field
• • None
• 0% - 25%
• Lifting: up to 10 pounds

Constantly

• Lifting: 10 to 25 pounds

Occasionally

• Lifting: 25 to 50 pounds

Rarely

• • 7 years of experience with data pipeline and workflow management tools (e.g., Apache Airflow, AWS Glue, Azure Data Factory)
• • 7 years of experience defining system architectures and exploring technical feasibility trade-offs for optimizing short term execution while planning for long term technical capabilities
• • 7 years of experience working with a variety of technology systems, designing solutions or developing data solutions in healthcare
• • 7 years of experience with cloud platforms (AWS, Azure, GCP) and their respective data services
• • 7 years of experience in data governance, data quality, and data security best practices
• • 7 years of experience translating requirements, design mockups, prototypes or user stories into technical designs
• • 7 years of experience in producing data-related code that is fault-tolerant, efficient, and maintainable
• • 7 years of experience structuring and transforming data from multiple source systems
• • 7 years of experience creating data visualizations and dashboards using Power BI or Tableau for internal and external audiences
• • 7 years of experience using Cribl as a steaming and transformation solution
• • 7 years of experience using Azure Data Factory as a transformation solution
• • 5 years of experience using Azure Data Explorer
• • Master's degree in Computer Science, Information Systems, Data Science, Computer Engineering or related field