Quanata is on a mission to help ensure a better world through context-based insurance solutions. They are seeking a Senior Data Engineer to help deliver data science services and build data pipelines that integrate data from multiple sources for risk evaluation and business intelligence.
Responsibilities:
- Design and build robust data pipelines that integrate data from diverse sources to support risk evaluation and business intelligence
- Build streaming data pipelines using Kafka and AWS services to enable real-time data processing and decision-making
- Create and operate data services that make curated datasets accessible to internal teams and external partners through APIs and scheduled deliveries
- Support data science workflows by building infrastructure that enables model training, feature engineering, and production inference
- Implement data quality frameworks including validation, monitoring, and alerting to maintain trust in our data assets
- Contribute to CI/CD pipelines and infrastructure-as-code practices to ensure reliable, repeatable deployments
- Deliver highly reliable services by following engineering best practices and participating in an on-call rotation to support production systems
- Contribute to the data warehouse by creating well-modeled datasets in Snowflake that empower analysts and data scientists
- Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver solutions that drive value
Requirements:
- Bachelor's degree or equivalent relevant experience
- 6-8 years of industry experience in data engineering or software engineering with a focus on data systems
- Strong proficiency in Python and SQL for data pipeline development and analysis
- Experience with AWS data platforms (Glue, S3, Athena, Lambda, Step Functions)
- Familiarity with Snowflake or similar cloud data platforms
- Familiarity with streaming technologies such as Kafka or Kinesis
- Experience with workflow orchestration tools (Airflow, Step Functions, or similar)
- Proficiency in infrastructure-as-code tools like Terraform
- Understanding of data modeling principles
- Excellent written and verbal communication with a strong collaborative focus
- Experience with CI/CD tailored for machine learning systems (e.g., automating model training, validation, and deployment)
- Prior experience working within the insurance industry or another highly regulated environment, demonstrating an understanding of pertinent regulatory, security, and data governance challenges
- Familiarity with ML infrastructure and supporting data science teams in productionizing models
- Experience designing and building feature stores or real-time feature serving systems
- Exposure to data governance and cataloging tools (e.g., DataHub, Atlan, or similar)