Contract_Lead Data Platform Engineer_Ontario, CA Hybrid_Local Candidates Only No Relocation

Job Title: Lead Data Platform Engineer

Location: Ontario, CA Hybrid
Requirement: Local Candidates Only No Relocation
Job Type: Contract

Job Summary

We are seeking an experienced Lead Data Platform Engineer to lead the design, development, optimization, and modernization of enterprise-scale data platforms. The ideal candidate will have strong hands-on expertise across the Hadoop ecosystem, Azure data services, and Databricks.

This role will provide technical leadership while working closely with data engineering, analytics, product, and infrastructure teams to build scalable, secure, and high-performing data solutions.

Key Responsibilities

• Lead the architecture, development, and enhancement of large-scale data platforms supporting analytics, machine learning, and operational workloads.
• Design, develop, and optimize big data solutions using Hadoop ecosystem technologies including HDFS, Hive, Spark, YARN, Oozie, Ranger, and Airflow.
• Develop and manage scalable data pipelines and transformations using Azure Data Lake Storage, Azure Data Factory, Azure Synapse, and Databricks.
• Implement and maintain data governance, security, data quality, and compliance frameworks across data platforms.
• Partner with data engineering, analytics, product, and infrastructure teams to translate business requirements into scalable technical solutions.
• Drive performance tuning, capacity planning, reliability, and cost optimization across on-premises and cloud data platforms.
• Provide technical leadership and mentorship to data engineers.
• Establish engineering standards, reusable patterns, development practices, and best practices.
• Oversee CI/CD pipelines, deployments, monitoring, and operational support for data workflows.
• Evaluate emerging technologies and contribute to enterprise data platform modernization and long-term technical strategy.

Required Qualifications

• 8+ years of experience in data engineering, data platforms, or related roles.
• 3+ years of experience in a technical lead, senior engineering, or architecture capacity.
• Strong hands-on experience with Hadoop ecosystem technologies including HDFS, Hive, Spark, YARN, Oozie, Ranger, and Airflow.
• Deep experience with Azure data services including:
• Azure Data Lake Storage (ADLS)
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Azure Functions
• Azure Key Vault
• Advanced experience with Databricks, including Spark optimization, Delta Lake, and Unity Catalog.
• Strong proficiency in Python and SQL.
• Experience with distributed data processing frameworks.
• Strong understanding of DevOps practices, CI/CD pipelines, and Infrastructure-as-Code technologies such as Terraform or ARM.
• Experience with data modeling and storage formats including Parquet, ORC, and Delta.
• Strong understanding of data governance, security, and data quality.
• Proven ability to lead technical teams and influence architecture and engineering decisions.
• Excellent communication and collaboration skills.

Preferred Qualifications

• Experience migrating on-premises Hadoop workloads to cloud platforms, preferably Azure Databricks.
• Experience with real-time data processing technologies such as Kafka, Azure Event Hubs, and Spark Streaming.
• Experience with enterprise data platform modernization initiatives.

Soft Skills

• Strong technical leadership and problem-solving abilities.
• Excellent communication and cross-functional collaboration skills.
• Ability to mentor and guide engineering teams.
• Ability to work effectively with technical and business stakeholders.
• Strong ownership and ability to drive projects from design through implementation.