Lead Data Engineer

Egen is hiring a Data Engineer to drive machine learning and data engineering initiatives in healthcare. The role designs, develops, supports, and documents GCP data pipelines and collaborates with project teams, business analysts, and project management to plan work and translate requirements into technical designs.

Responsibilities:

• Design, architect, develop, support GCP data pipelines to extract, load, and transform data
• Maintain a holistic view of information assets by creating and maintaining artifacts that illustrate how information is stored, processed, and supported (i.e. documentation)
• Work with the project team, estimate and plan the work effort
• Attend daily team standups
• Work with business analysts and project management to review business requirements and and produce technical design specs that will meet the requirements

Requirements:

• 5+ years of hands-on experience as a Data Engineer or Data Architect
• Proven track record leading technical projects and teams
• Expert proficiency in Google Cloud Platform (GCP) tools, including: Google Cloud Storage (GCS), BigQuery, Cloud Composer/Airflow, Dataproc/Spark
• Strong data analysis and problem-solving skills
• Proficiency in a Business Intelligence (BI) tool
• Advanced SQL skills, including writing, tuning, and interpreting complex SQL queries
• Experience writing and maintaining Unix/Linux shell scripts
• Solid understanding of Data Build Tool (DBT)
• Experience with CI/CD pipelines
• Machine Learning experience
• Proficiency in developing Python-based ELT data pipelines
• Expertise in optimizing GCP BigQuery SQL queries and scripts