Sr. Data Engineer with Databricks(Need Locals)

We are seeking an experienced Senior Data Engineer with strong Databricks expertise to join our data engineering team supporting in Houston, TX. The ideal candidate will have hands-on experience designing and developing scalable data pipelines, data platforms, and ETL/ELT solutions using Databricks, Apache Spark, Python, and SQL.

The candidate will work closely with data architects, analysts, application teams, and business stakeholders to build reliable and high-performance data solutions supporting enterprise analytics and data-driven initiatives.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes using Databricks.
• Develop data engineering solutions using Databricks, Apache Spark, PySpark, Python, and SQL.
• Build and optimize data pipelines using Delta Lake and Databricks Lakehouse architecture.
• Work with large and complex datasets and implement efficient data transformation and processing strategies.
• Develop batch and near-real-time data ingestion pipelines from multiple enterprise data sources.
• Implement data quality, validation, monitoring, and error-handling processes.
• Optimize Spark jobs, SQL queries, data models, and Databricks workloads for performance and cost.
• Work with cloud-based data platforms and integrate Databricks with enterprise systems.
• Collaborate with data architects and engineering teams on data architecture and solution design.
• Implement CI/CD and automated deployment processes for data engineering workloads.
• Troubleshoot production data pipelines and resolve performance and data-quality issues.
• Participate in code reviews, technical documentation, and engineering best practices.
• Ensure solutions meet enterprise security, governance, scalability, and reliability standards.

Required Skills

• 8+ years of experience in Data Engineering
• Strong hands-on experience with Databricks
• Strong experience with Apache Spark / PySpark
• Strong programming skills in Python
• Advanced SQL skills
• Experience developing enterprise-scale ETL/ELT data pipelines
• Strong knowledge of Delta Lake / Lakehouse architecture
• Experience with data warehousing and data modeling concepts
• Experience working with large-volume structured and unstructured data
• Experience with cloud data platforms such as AWS, Azure, or Google Cloud Platform
• Experience with Git and CI/CD practices
• Strong troubleshooting and performance-tuning skills

Preferred Qualifications

• Experience with Databricks Workflows, Unity Catalog, and Delta Live Tables (DLT/Lakeflow Declarative Pipelines)
• Experience with Azure Databricks
• Experience with AWS Databricks
• Experience with Kafka or other streaming technologies
• Experience with REST APIs and enterprise data integrations
• Experience with Terraform or infrastructure-as-code
• Knowledge of data governance, security, lineage, and access-control frameworks
• Experience working in large enterprise environments
• Experience in the Oil & Gas / Energy industry is highly preferred
• Experience supporting data analytics, reporting, or AI/ML data platforms is a plus

Education

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
• Equivalent professional experience may be considered.