Lead Data Engineer

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world. They are seeking a Lead Data Engineer who will be responsible for defining and deploying end-to-end data architecture solutions, leading teams, and extracting value from complex data sets.

Responsibilities:

• 10+ Years of Experience in Data Engineering
• At least four years of experience designing and delivering data engineering solutions with Databricks
• Ability to independently define and deploy an end to end data architecture that includes Databricks medallion architecture
• Experience leading large teams of engineers and developers to deploy Databricks solutions
• Expert level hands on development knowledge in PySpark/Python
• Expert level hands on SQL development skills
• Hands on cloud platform experience in two of the following cloud platforms: Azure, AWS, Google Cloud
• Prior expertise deploying Delta Lake Solutions into production
• Experience extracting value from large and complex sets of data from various sources and databases
• Solid grasp of database engineering and design principles
• Familiarity with CI/CD methods desired
• Background working with orchestration tools, such as Airflow
• Strong knowledge of Unity Catalog
• Past or current experience with an ETL tools such as Infomatica, Talend, Matillion desired
• Experienced in managing senior client stakeholders
• Delta Sharing and Marketplace Expertise
• Experience working with Databricks Apps
• Working knowledge of Databricks Genie, AI/BI and Databricks Data Visualization Capabilities
• Conceptual, Logical and

Requirements:

• 10+ Years of Experience in Data Engineering
• At least four years of experience designing and delivering data engineering solutions with Databricks
• Ability to independently define and deploy an end to end data architecture that includes Databricks medallion architecture
• Experience leading large teams of engineers and developers to deploy Databricks solutions
• Expert level hands on development knowledge in PySpark/Python
• Expert level hands on SQL development skills
• Hands on cloud platform experience in two of the following cloud platforms: Azure, AWS, Google Cloud
• Prior expertise deploying Delta Lake Solutions into production
• Experience extracting value from large and complex sets of data from various sources and databases
• Solid grasp of database engineering and design principles
• Familiarity with CI/CD methods desired
• Background working with orchestration tools, such as Airflow
• Strong knowledge of Unity Catalog
• Past or current experience with an ETL tools such as Infomatica, Talend, Matillion desired
• Experienced in managing senior client stakeholders
• Delta Sharing and Marketplace Expertise
• Experience working with Databricks Apps
• Working knowledge of Databricks Genie, AI/BI and Databricks Data Visualization Capabilities