Senior Data Engineer

Commence is at the forefront of data-centric transformation in healthcare, aiming to enhance health outcomes through efficient data solutions. The Senior Data Engineer will design and maintain automated data pipelines, ensuring data quality and supporting analytics across the organization.

Responsibilities:

• Design, develop, and maintain scalable data pipelines to collect, process, and transform data from various sources
• Integrate data from multiple sources, ensuring data quality and consistency across the organization
• Build and maintain data storage solutions, including data warehouses and data lakes, ensuring optimal performance and reliability
• Implement data transformation and enrichment processes to prepare data for analytics and reporting
• Leverage cloud technologies, particularly AWS, to optimize and manage data infrastructure
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions
• Create and maintain comprehensive documentation for data pipelines, data models, and related processes
• Mentors and guides junior data engineers/ analysts on data engineering best practices and industry standards
• Other duties as assigned

Requirements:

• Minimum of 4 years of experience in data engineering or a related field
• Strong experience with data pipeline/ orchestration and ETL development using tools such as Apache Airflow, Kubernetes, Databricks Workflows or similar
• Demonstrated experience in designing highly efficient programs capable of processing terabytes of data
• Strong Proficiency in SQL and experience with relational databases (e.g., SQLServer, PostgreSQL) and NoSQL databases (e.g., MongoDB, OpenSearch)
• Experience with cloud technologies, particularly AWS (e.g., S3, Redshift, Glue, Lambda, Athena)
• Proficient in writing data programs in R, Python, Scala, or similar language
• Familiarity with big data technologies such as Apache Spark, Databricks, or similar
• Familiarity with data visualization tools and data migration methods
• Excellent problem-solving skills and attention to detail
• Strong communication and interpersonal skills, with the ability to work effectively with diverse teams and stakeholders
• Bachelor's degree in computer science, Information Technology, or a related field
• Familiarity with data governance and data quality best practices is a plus
• Familiarity with healthcare data standards i.e. (FHIR, HL7)
• Familiarity working with unstructured data i.e. pdfs, free-text, etc
• Databricks Data Engineering certifications
• Data Visualization/ Reporting skills (i.e. PowerBI, Tableau, or Quicksight)