Convey Health Solutions is committed to delivering healthcare data products and platform components. They are seeking a Lead Data Engineer to oversee the development and continuous improvement of their data engineering solutions while ensuring compliance with industry standards.
Responsibilities:
- Lead the creation of automated data validation frameworks to ensure client data integrity is at the level needed for our solutions
- Follow best practices and coding standards to create custom scripts to clean/map/align client data to the expected inputs for the ETL pipelines of various solutions
- Architect and lead the adoption of serverless data platforms, integrating services like AWS Glue, Lambda, DynamoDB, and Lakehouse Formation
- Lead GitOps strategy for infrastructure and pipeline deployment, ensuring compliance, rollback safety, and peer-reviewed workflows
- Establish best practices for feature flag toggling strategies in critical data pipelines
- Drive internal knowledge sharing and technical writing culture, including mentoring junior staff on documentation standards
- Thrives in collaborative team environments and takes pride in mentoring junior team members by providing guidance, support, and best practices
- Adept at designing scalable architectural solutions while effectively communicating complex concepts to both technical and non-technical audiences
- Skilled in tailoring messaging and presentation style based on audience needs to ensure clarity, alignment, and impact across stakeholders
- Owns technical environment and tooling strategy across the data engineering organization. This role ensures standards, documentation, and collaboration practices are in place and consistently applied
Requirements:
- Bachelor's degree in computer science, Engineering, or a related field of study, or equivalent experience
- 10+ years in data engineering roles
- Extensive experience working with large-scale data systems and cloud-based architectures (AWS)
- Python, PySpark, Scala (10+ yrs) – Directed large-scale data platform development with an emphasis on modularity and performance
- Airflow (6+ yrs) – Defined standards for DAG design, observability, and reusability across the team
- AWS (Glue, EMR, Lambda, Step Functions) (5+ yrs) – Oversaw hybrid orchestration strategies using serverless and EMR-based pipelines
- Iceberg (4–5 yrs) – Led lakehouse strategy including governance, versioning, and cost optimization
- Deep expertise in designing and optimizing data lake and lakehouse architectures at scale, including governance, schema evolution, cost efficiency, and access control
- Leads strategy and implementation of hybrid lakehouse ecosystems leveraging technologies like Iceberg, Delta Lake, and catalog services across multiple environments (e.g., dev, staging, prod)
- Directed team of data engineers of 5–8 engineers, driving architecture decisions, enforcing engineering standards, and championing Agile delivery practices within a high-performance data engineering environment
- Docker/Kubernetes (6–8 yr) – Oversaw Kubernetes-based orchestration for enterprise data pipelines, established Docker standards across teams and integrated container practices with compliance and DevOps strategies
- Excellent written, verbal, and interpersonal communication skills
- Excellent organizational skills and attention to detail
- Excellent documentation skills
- Team player
- Ability to provide clear and accurate information through multiple media
- Ability to extrapolate, from historical trends, future volumes, and staffing needs
- Ability to manage open requests and follow up when necessary, without outside direction
- Ability to manage time effectively with strong attention to detail
- Ability to read and interpret documents including safety rules, operating and maintenance instructions, procedure manuals and general correspondence
- Ability to write routine reports and correspondence
- Ability to deal with problems involving several concrete variables in standardized situations
- Ability to interact politely, tactfully, and firmly with a wide range of people and personalities
- Ability to work in an environment with potential interruptions
- Ability to manage multiple simultaneous tasks with individual timeframes and priorities
- Ability to motivate and encourage a team of front-line employees
- Ability to set and manage to expectations for a team