Data Engineer

Role Overview

• Design, develop, and maintain end-to-end data pipelines using Databricks (Spark, PySpark, SQL) to ingest, transform, and curate structured and semi-structured data from authoritative DoD source systems.
• Engineer and optimize data models, Delta tables, and curated datasets to support financial reporting, PPBE workflows, auditability, and mission analytics within Advana and downstream Foundry use cases.
• Develop, maintain, and deploy Foundry pipelines, transforms, and data products, enabling traceability, lineage, and repeatable analytics workflows for DoD stakeholders.
• Implement and manage CI/CD and version-controlled data engineering workflows using Git-based repositories and automated deployment patterns aligned with DoD security and ATO requirements.
• Build and deploy cloud-native solutions in secure AWS and Azure environments, supporting IL4/IL5 architectures as applicable, and leveraging containerization where approved.
• Partner with functional SMEs, financial analysts, platform engineers, and governance teams to ensure data pipelines conform to DoD financial data standards, business rules, and controls.
• Implement robust data quality, reconciliation, validation, and monitoring capabilities to support financial accuracy, audit readiness, and confidence in reported metrics.
• Support data integration, modernization, and migration efforts, including onboarding legacy financial systems and transitioning workloads to Databricks and Foundry within Advana.
• Troubleshoot and resolve pipeline, performance, and data integrity issues in production environments, ensuring high availability and reliability in mission-critical systems.
• Document data flows, engineering designs, and operational procedures to support compliance, knowledge transfer, and platform accreditation.
• Stay current on emerging data engineering, analytics platform, and DevSecOps best practices relevant to DoD enterprise data environments.

Requirements

• US Citizenship is required.
• Must have an ACTIVE and MAINTAINED "SECRET" Federal or DoD security clearance with the ability to meet all client security and access requirements.
• Bachelor’s degree obtained
• Minimum FIVE (5) years of experience in data engineering, analytics engineering, or software development, preferably in regulated or federal environments.
• Strong proficiency in Python, SQL, PySpark, and Spark-based data processing.
• Demonstrated hands-on experience with Databricks, including Delta Lake, notebook development, performance tuning, and production pipelines; experience supporting or integrating with DoD Advana is highly desirable.
• Experience building or supporting data solutions using Palantir Foundry, including pipelines, transforms, datasets, and data lineage.
• Experience working with relational and analytical data stores (e.g., Databricks, Oracle, Postgres, SQL Server) and applying sound data modeling principles.
• Familiarity with Git-based version control, CI/CD concepts, and Agile delivery methodologies in secure environments.
• Experience handling federal financial data, including budget, execution, and accounting data, with an understanding of data controls and reconciliation needs.
• Ability to operate independently in a fast-paced, mission-focused environment.
• Strong analytical, troubleshooting, and communication skills, including the ability to translate complex technical concepts for both technical and non-technical audiences.

Tech Stack

• AWS
• Azure
• Cloud
• Oracle
• Postgres
• PySpark
• Python
• Spark
• SQL

Benefits

• Medical, Rx, Dental & Vision Insurance
• Personal and Family Sick Time & Company Paid Holidays
• Position may be eligible for a discretionary variable incentive bonus
• Parental Leave and Adoption Assistance
• 401(k) Retirement Plan
• Basic Life & Supplemental Life
• Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
• Short-Term & Long-Term Disability
• Student Loan PayDown
• Tuition Reimbursement, Personal Development & Learning Opportunities
• Skills Development & Certifications
• Employee Referral Program
• Corporate Sponsored Events & Community Outreach
• Emergency Back-Up Childcare Program
• Mobility Stipend