Lead Cloud Data Engineer – Multi-Cloud Data Platforms

Role Overview

• Design, build, and maintain cloud-native data pipelines and data products across Azure and AWS using Databricks and Snowflake.\n
• Lead and contribute to the modernization and migration of on‑prem and legacy data platforms to cloud-based solutions.\n
• Implement batch and streaming data processing patterns using Spark and cloud-native services.\n
• Partner with data governance, security, and risk teams to ensure data products comply with enterprise governance, data privacy, and regulatory requirements.\n
• Enable secure data sharing and access patterns across domains and platforms using appropriate controls.\n
• Define and promote data engineering best practices, including CI/CD, testing, observability, performance tuning, and cost optimization.\n
• Collaborate with product owners and analytics teams to translate business requirements into well-modeled, high-quality datasets.\n
• Work closely with cloud and security architects to implement secure, scalable, and resilient data solutions.\n
• Support and mentor junior engineers through design reviews, code reviews, and technical guidance.

Requirements

• Bachelor’s degree, or equivalent work experience\n
• Six to eight years of relevant experience\n
• Experience with data architecture and platform design in large enterprises.\n
• Knowledge of data sharing, data mesh, or domain-driven data architecture concepts.\n
• Strong problem-solving skills and a track record of delivering scalable, efficient data solutions.\n
• Strong hands-on experience with Azure Data Platform services, including: Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics (or Fabric equivalent experience)\n
• Experience with AWS data services, such as AWS Glue, S3, and event-driven integrations.\n
• Deep experience with Databricks (Spark, Delta Lake, performance tuning).\n
• Strong working knowledge of Snowflake, including data modeling, ingestion patterns (e.g., Snowpipe), and data sharing.\n
• Expertise in Apache Spark for large-scale data processing.\n
• Experience building batch and near-real-time data pipelines.\n
• Strong SQL skills and experience with dimensional and analytical data modeling.\n
• Experience designing reusable, domain-oriented data products.\n
• Experience with API-based integrations (REST; familiarity with SOAP and GraphQL is a plus).\n
• Hands-on experience integrating with API gateways.\n
• Understanding of messaging and streaming platforms such as Kafka, MQ, AWS SQS, or RabbitMQ.\n
• Strong understanding of IAM, RBAC, OAuth 2.0, TLS/mTLS, and JWT.\n
• Experience implementing secure data access patterns in cloud environments.\n
• Familiarity with data cataloging, lineage, and metadata management concepts.\n
• Experience enabling self-service analytics and BI using tools such as Power BI, Tableau, or equivalent.\n
• Support AI initiatives through the data platform and data products.\n
• Prior experience in financial services or other highly regulated industries.\n
• Professional certifications in Microsoft Azure and/or AWS.\n
• Master’s degree in a relevant technical field.

Tech Stack

• Apache
• AWS
• Azure
• Cloud
• GraphQL
• Kafka
• RabbitMQ
• SOAP
• Spark
• SQL
• Tableau

Benefits

• Healthcare (medical, dental, vision)\n
• Basic term and optional term life insurance\n
• Short-term and long-term disability\n
• Pregnancy disability and parental leave\n
• 401(k) and employer-funded retirement plan\n
• Paid vacation (from two to five weeks depending on salary grade and tenure)\n
• Up to 11 paid holiday opportunities\n
• Adoption assistance\n
• Sick and Safe Leave accruals of one hour for every 30 worked, up to 80 hours per calendar year unless otherwise provided by law