GCP Lead Data Engineer Grade C2

Capgemini is seeking a GCP Lead Data Engineer to support its DACO team with data pipeline and ingestion needs. The role involves building and optimizing cloud-based ETL/ELT pipelines, managing GCP data infrastructure, leading engineering practices, and supporting analytics and machine learning use cases.

Responsibilities:

• Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions
• Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage
• Team Leadership: Mentor junior engineers, review code, and establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging)
• Performance & Cost: Optimize pipeline performance, query efficiency, and GCP cost management (e.g., slot reservations, partitioning)
• Collaboration: Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases
• Data Quality & Governance: Implement data validation, lineage, and metadata management (e.g., Data Catalog, dbt)

Requirements:

• Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam)
• Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform
• CloudSQL, Cloud Storage and GCP Suite, Cloud Run (Heavy) and custom templates
• Kafka (based upon data source) Data Plex is use with Every workflow
• Experience leading projects, mentoring teams, and driving technical roadmaps
• Familiarity with CI/CD, Git, and infrastructure-as-code
• GCP Certifications (e.g., Professional Data Engineer)
• Experience with real-time processing, Vertex AI, or multi-cloud environments
• DBT Labs & understanding of clinical/CMS billing data (context for latter: resources will be working with full medallion layers in cloud.)