Marsh McLennanRemote
Senior Data Engineer
United States of America
Full Time
1 hour ago
$96,000 - $168,000 USD
H1B Sponsor Likely
Key skills
Data EngineeringAzure Cloud ServicesAzure DatabricksAzure FunctionsAzure Data FactoryApache SparkPySparkPythonDatabricks SQLQuery OptimizationETL/ELT ProcessesData Pipeline DevelopmentWorkflow OrchestrationDelta LakeChange Data CaptureACID TransactionsData ModelingNormalized Data ModelingDimensional Data ModelingLakehouse Data ModelingBatch Data ProcessingReal-time Data ProcessingStreaming Data ProcessingKafkaEvent HubData ArchitectureDistributed SystemsCloud-native Design PatternsData QualityData IntegrityData SecurityAIMachine Learning DeploymentDatabricks Agents BricksMCP ServersDatabricks Genie WorkspacesConversational AnalyticsCI/CD ToolsAzure DevOpsGitInfrastructure as CodeTerraformARM TemplatesData GovernanceData CatalogingUnity CatalogAttribute-based Access ControlRole-based Access ControlMachine Learning WorkloadsBusiness Intelligence WorkloadsInsurance Data DomainHealthcare Data DomainDatabricks CertificationData Engineering CertificationAzure Cloud CertificationScalaSQLNoSQLMachine LearningETLELTData WarehousingAnalyticsBusiness IntelligenceDatabricksSparkAzureSQL ServerCI/CDCollaboration
About this role
Marsh McLennan Agency is a leading provider of business insurance and employee health solutions. They are seeking a Senior Data Engineer to develop and maintain scalable data pipelines and enhance data models, utilizing Azure cloud solutions and various programming languages to drive data-driven decision-making in the organization.
Responsibilities:
- Develop and maintain scalable data pipelines
- Enhance data models
- Provide subject matter expertise in data acquisition and consumption pipelines in Azure cloud solutions
- Develop best practices, reusable code, libraries, and frameworks for cloud-based data warehousing and ETL
- Use multi-cloud, programming languages like Scala, Python, SQL, in Unity Catalog, SQL Server and other RDBMS, NoSQL databases
- Design enterprise data warehouse platforms
- Foster collaboration and align with business objectives
- Elevate data models
- Drive data-driven decision-making
- Enhance data accessibility throughout the organization
Requirements:
- Minimum of 10 years of practical data engineering experience in enterprise settings
- Strong proficiency with Azure services, particularly Azure Databricks, Azure Functions, and Azure Data Factory (preferred)
- Advanced skills in Apache Spark using PySpark, Python and Databricks SQL, including query optimization and performance tuning
- In-depth knowledge of ETL/ELT processes, data pipeline development, and orchestration/scheduling and workflows
- Hands-on experience with Delta Lake features such as Change data capture, ACID transactions, optimization, and schema drifting/evolution
- Solid background in data modeling techniques, including normalized, dimensional, and Lakehouse models
- Experience handling both batch and real-time/streaming data processing using technologies like Kafka or Event Hub
- Strong understanding of data architecture principles, distributed systems, and cloud-native design patterns
- Ensure data quality, integrity, and security throughout the data lifecycle
- Build, deploy and manage AI and machine learning applications on Databricks using Agents Bricks and custom MCP servers
- Build and deploy Databricks Genie workspaces and enable conversational analytics on custom datasets
- Familiarity with CI/CD tools such as Azure DevOps and Git
- Experience with Infrastructure as Code (IaC) tools like Terraform and ARM templates
- Knowledge of data governance and cataloging solutions, including Unity Catalog for enabling Attribute and Role based access controls
- Experience supporting machine learning or business intelligence workloads on Databricks
- Exposure to Insurance and healthcare data domains is a huge plus
- Certifications in Databricks, data engineering and Azure cloud is a plus