Build and optimize distributed data processing jobs using Apache Spark on Databricks.
Implement Delta Lake, DLT pipelines, dbt transformations and Medallion architecture for scalable and reliable data workflows.
Design and automate ETL pipelines using Azure Data Factory, Databricks, and Synapse Analytics.
Integrate data from diverse sources including Duck Creek, Intacct, Workday and external APIs.
Develop dimensional models (Star/Snowflake schemas), stored procedures, and views for data warehouses.
Ensure efficient querying and transformation using SQL, T-SQL, and PySpark.
Leverage Azure DevOps, CI/CD pipelines, and GitHub for version control and deployment.
Utilize Azure Logic Apps and ML Flow for workflow automation and model training.
Implement role-based access control (RBAC), data encryption, and auditing mechanisms.
Work closely with data scientists, analysts, and business stakeholders to deliver high-quality data solutions.
Mentor junior engineers and contribute to code reviews and architectural decisions.

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
5+ years of experience in data engineering with at least 2 years on Databricks.
Proficiency in Python, Scala, SQL, and Spark.
Hands-on experience with Azure Data Services (ADF, ADLS, Synapse).
Strong understanding of ETL, data warehousing, and data modeling concepts.
Experience with Microstrategy, Power BI, including DAX and advanced visualizations.
Familiarity with MLflow, LangChain, and LLM integration is a plus.
Knowledge of Duck Creek a plus.
Insurance Domain knowledge preferred.
Preferred Certifications Databricks Data Engineering Professional Azure/AWS Data Engineering Certifications

Opportunities to stretch and grow: your professional and personal development matters to us.
Clarity and kindness: you can rely on us to be open, honest and supportive, offering clarity on what success looks like.
Support in good times and bad: we believe in showing up for each other consistently, not only when it’s easy.
A community that cares: we are committed to sustaining a community in which each person feels cared for as an individual.

Data Engineering Lead

Key skills