Lead Data Solution and AI Engineer

Gallagher is hiring a Lead Data Solution and AI Engineer to guide and deliver data integration solutions for business intelligence and AI initiatives. The role leads technically complex data projects, builds and maintains data pipelines and AI/ML capabilities, and provides architectural guidance and leadership to project teams and junior developers.

Responsibilities:

• Design and maintain scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, Synapse Analytics, and Azure SQL to support analytics, reporting, and AI/ML workloads
• Develop data ingestion, transformation, validation, and loading workflows from APIs, databases, Azure Blob Storage, ADLS Gen2, third-party systems, and on-premises SQL databases
• Support on-premises SQL Server integration, including data extraction, synchronization, replication, and performance tuning
• Write and optimize T-SQL queries, stored procedures, views, and functions to support data processing, reporting, and analytics
• Support data models, data marts, and feature stores for analytics and machine learning use cases
• Develop, train, fine-tune, deploy, and monitor AI/ML models using structured and unstructured data, including model versioning, performance tracking, and retraining
• Develop AI-powered capabilities using modern large language models (LLMs) and AI services
• Implement intelligent search, knowledge retrieval, and automation solutions
• Troubleshoot ETL failures, data latency issues, and production incidents across Azure and on-premises environments
• Identify and implement process improvements that automate manual activities and improve data delivery
• Improve the performance, scalability, reliability, and maintainability of data engineering solutions
• Apply Azure security best practices, including RBAC, Managed Identities, Azure Key Vault, and private networking
• Ensure data quality, consistency, lineage, and governance through validation, reconciliation, and exception-handling processes
• Provide technical guidance and leadership to junior developers
• Provide data architecture recommendations, project team leadership, technology vision, and detailed estimates

Requirements:

• 10+ years of progressive experience as a Data Solutions and AI Engineer using current and emerging data technologies and approaches, including 7+ years on the Microsoft platform
• 7+ years designing and building large applications using SQL Server and related database technologies
• 7+ years experience with physical and cloud database design
• 7+ years experience migrating code, database changes, data, and menus through the software development lifecycle
• 2+ years experience designing, developing, and deploying AI/ML-powered applications and cloud data solutions
• 2+ years experience with NLP, entity resolution algorithms, Graph RAG, ontology, and distributed systems
• 4+ years experience with CI/CD pipelines and deployment tools
• 2+ years experience developing search, retrieval, recommendation, or knowledge management solutions
• 2+ years experience with microservices, event-driven architectures, and containerized applications
• 3+ years experience working with C#, .NET, REST APIs, and modern application architecture to support integration and solution development
• 5+ years strong understanding of database and application performance tuning
• Bachelor's degree in computer science, Engineering, or a related field
• Strong communication skills with management, development teams, vendors, systems administrators, and other service providers
• Typically requires a University Degree and minimum 7 years of prior relevant experience, or equivalent experience
• You are a strong communicator and an active listener
• A passion for technology
• Deep technical understanding of all things data
• Ability to lead a project team
• Ability to paint a technology vision and communicate its value
• Ability to create detailed estimates for planned technology activities
• A disciplined approach to your work life and your craft
• Stay current with emerging data engineering and AI technologies
• Experience in Snowflake and Microsoft Fabric a plus
• SQL Python, Azure data factory, Data
• Bricks, Spark (experience would be a plus)
• Data transformation
• Cloud Data Experience
• Distributed data warehouse knowledge
• Data Lakes
• Communication and stakeholder management skills