Key skills
Azure DatabricksAzure FunctionsAzure Data FactoryApache SparkPySparkPythonDatabricks SQLSQLETL/ELT processesData pipeline developmentDelta LakeData modelingKafkaEvent HubData architectureDistributed systemsCloud-native design patternsData quality managementData securityAI application deploymentMachine learning application deploymentDatabricks Genie workspacesConversational analyticsAzure DevOpsGitTerraformARM templatesData governanceData catalogingUnity CatalogInsurance data domainHealthcare data domainDatabricks certificationData engineering certificationAzure cloud certificationStakeholder ManagementScalaNoSQLAIMachine LearningETLELTData EngineeringData WarehousingAnalyticsBusiness IntelligenceDatabricksSparkAzureSQL ServerCI/CDCollaboration
About this role
Marsh McLennan Agency (MMA) provides business insurance, employee health & benefits, retirement, and private client insurance solutions to organizations and individuals seeking limitless possibilities. As a Senior Data Engineer on the MMA National IT Team, you will develop and maintain scalable data pipelines, enhance data models, and provide subject matter expertise in data acquisition and consumption pipelines in Azure cloud solutions.
Responsibilities:
- Develop and maintain scalable data pipelines
- Enhance data models
- Provide subject matter expertise in data acquisition and consumption pipelines in Azure cloud solutions including and not limited to Databricks, ADF and other ETL/ELT tools
- Develop best practices, reusable code, libraries, and frameworks for cloud-based data warehousing and ETL
- Use multi-cloud, programming languages like Scala, Python, SQL, in Unity Catalog, SQL Server and other RDBMS, NoSQL databases, and design enterprise data warehouse platforms
- Foster collaboration and align with business objectives to elevate data models, drive data-driven decision-making, and enhance data accessibility throughout the organization
Requirements:
- Minimum of 10 years of practical data engineering experience in enterprise settings
- Strong proficiency with Azure services, particularly Azure Databricks, Azure Functions, and Azure Data Factory
- Advanced skills in Apache Spark using PySpark, Python and Databricks SQL, including query optimization and performance tuning
- In-depth knowledge of ETL/ELT processes, data pipeline development, and orchestration/scheduling and workflows
- Hands-on experience with Delta Lake features such as Change data capture, ACID transactions, optimization, and schema drifting/evolution
- Solid background in data modeling techniques, including normalized, dimensional, and Lakehouse models
- Experience handling both batch and real-time/streaming data processing using technologies like Kafka or Event Hub
- Strong understanding of data architecture principles, distributed systems, and cloud-native design patterns
- Ensure data quality, integrity, and security throughout the data lifecycle
- Build, deploy and manage AI and machine learning applications on Databricks using Agents Bricks and custom MCP servers
- Build and deploy Databricks Genie workspaces and enable conversational analytics on custom datasets
- Familiarity with CI/CD tools such as Azure DevOps and Git
- Experience with Infrastructure as Code (IaC) tools like Terraform and ARM templates
- Knowledge of data governance and cataloging solutions, including Unity Catalog for enabling Attribute and Role based access controls
- Experience supporting machine learning or business intelligence workloads on Databricks
- Exposure to Insurance and healthcare data domains is a huge plus
- Certifications in Databricks, data engineering and Azure cloud is a plus