Databricks Data Engineer - Clearance Required

The Squires Group is seeking a Senior Databricks Data Engineer for a federal technology modernization program. The role builds and supports Databricks data pipelines and migrates data and business logic from legacy systems to cloud-based environments.

Responsibilities:

• Build, operate, and maintain ingestion pipelines within Databricks
• Develop and support Bronze, Silver, and Gold data pipelines using Medallion Architecture
• Migrate data and business logic from legacy systems into modern cloud-based environments
• Gather and analyze technical and business requirements
• Design and develop batch, incremental, and streaming ingestion pipelines
• Monitor data jobs and troubleshoot pipeline and production issues
• Provide post-go-live production support and break/fix assistance
• Document technical processes, data workflows, and procedures
• Collaborate with technical teams and stakeholders in an Agile environment

Requirements:

• An active DoD Secret clearance and U.S. citizenship are required for this position
• Strong experience in data engineering, ETL/ELT development, or data warehousing
• Strong PySpark and/or Spark SQL skills with solid SQL fundamentals
• Experience with Databricks and Medallion Architecture, including Bronze, Silver, and Gold layering
• Experience with Delta Lake, including ACID transactions, schema evolution, time travel, and MERGE/upsert patterns
• Experience designing batch and incremental/streaming ingestion pipelines
• Familiarity with Agile processes
• Excellent written and verbal communication skills
• Strong interpersonal, organizational, and prioritization skills
• Databricks certification or additional hands-on Databricks experience preferred
• DoD, Navy, or other federal experience preferred
• AWS GovCloud experience preferred
• Experience migrating code from non-accredited to accredited AWS environments is a plus