Data Cloud Engineer

Zachary Piper Solutions is seeking a Data Cloud Engineer to support a long-term federal modernization program focused on enterprise cloud data solutions and mission-critical analytics in secure DoD environments. The role modernizes enterprise data infrastructure and enables secure, scalable data integration and reliable access to data for analytics, reporting, and operational decision-making.

Responsibilities:

• Design, develop, and maintain production-grade ETL/ELT pipelines, with a strong emphasis on hands-on, high-performance PySpark development and Spark SQL optimization
• Build and maintain scalable data pipelines using Azure Synapse Analytics, Databricks, AWS Glue, and related cloud technologies
• Develop ingestion frameworks for structured and unstructured data, including relational databases, flat files, JSON, nested data, and REST APIs
• Implement full and incremental data loads, including CDC, late-arriving records, and rerunnable pipeline architectures
• Design and maintain cloud data lakes, warehouses, and analytics-ready datasets supporting enterprise reporting and operational decision-making
• Implement data quality and governance controls, including schema validation/enforcement, schema drift, RBAC, lineage, cataloging, and credential management
• Monitor and troubleshoot pipelines for performance, latency, failures, logging, alerting, and operational reliability
• Support CI/CD, automated deployments, rollback strategies, and environment promotions across cloud data environments
• Collaborate with cybersecurity, cloud, DevOps, and software engineering teams to support RMF, STIG, FedRAMP, and DoD security requirements, data migrations, integrations, and Agile delivery

Requirements:

• 10+ years of recent experience designing and operating scalable, production-grade data pipelines using Azure Synapse and/or Databricks, w/ a Bachelors degree in Computer Science, Data Science, Engineering, Information Systems, or a related field
• Strong hands-on PySpark development experience is required, including demonstrated high-performance coding skills developing ETL pipelines; candidates without this experience will not be considered
• Advanced experience with Spark SQL, Python, and SQL, including large-scale data transformation and optimization
• Experience ingesting structured and unstructured data and implementing full/incremental loads, CDC, and rerunnable pipeline architectures
• Strong experience with data quality, schema validation/enforcement, pipeline monitoring, logging, alerting, and troubleshooting
• Experience implementing CI/CD practices and automated deployments for data pipelines and cloud environments
• Experience with data governance, RBAC, and credential management in secure cloud environments
• Hands-on experience with Azure and/or AWS data services, including Azure Synapse, Azure Data Factory, Databricks, AWS Glue, Redshift, and S3
• Active Secret clearance