Job Title: Senior Databricks Data Engineer - Remote / Telecommute
Job Location: Aliso Viejo, CA
Job Type: Contract
Pay Range: $56.00hr - $61.00hr
Job Overview:
The organization is seeking an experienced Senior Databricks Data Engineer with strong expertise in designing, developing, and optimizing large-scale data engineering solutions on the Databricks platform. The ideal candidate will have extensive hands-on experience in building enterprise-grade data pipelines, processing high-volume datasets, and delivering scalable analytics solutions using PySpark, Spark SQL, and AWS cloud technologies.
Responsibilities:
The candidate will design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, Spark SQL, and Apache Spark. The successful candidate will build and optimize end-to-end ETL/ELT pipelines integrating Databricks with AWS S3, Snowflake, Airflow, and AWS Glue.
The candidate s responsibilities include processing and transforming large-scale structured and semi-structured datasets to support enterprise reporting, analytics, and regulatory requirements. The candidate will develop reusable frameworks and components for data ingestion, cleansing, transformation, validation, reconciliation, enrichment, and business rule implementation.
The successful candidate will implement robust data quality checks, monitoring, and governance controls to ensure data accuracy and reliability. The candidate will perform performance tuning and optimization of Spark applications and Databricks workloads to improve scalability and processing efficiency.
The candidate will collaborate with business stakeholders, architects, and analytics teams to understand data requirements and deliver scalable solutions. The successful candidate will troubleshoot and resolve production data pipeline issues while ensuring high availability and reliability, supporting advanced analytics, customer segmentation, campaign analytics, AML/regulatory reporting, and data validation initiatives.
Technical Skills:
Preferred Skills:
Key Experience Areas:
The candidate should have demonstrated experience in one or more of the following:
====
