Spark Data Engineer (Remote)

Zachary Piper Solutions is seeking a Spark Data Engineer to support a variety of federal customers. This role focuses on big data engineering, optimization, and performance tuning using Apache Spark, requiring strong data engineering skills and excellent communication for client collaboration.

Responsibilities:

• Design and implement scalable data pipelines and architectures using Apache Spark
• Optimize Spark jobs for performance and efficiency across large datasets
• Collaborate with clients to understand requirements and deliver technical solutions
• Support data engineering tasks including ETL, data warehousing, and query optimization
• Assist with integration of Spark-based solutions into cloud environments (AWS, Azure, GCP)
• Provide technical guidance on best practices for big data processing and analytics
• Participate in short- to medium-term customer engagements, ensuring successful delivery of projects

Requirements:

• Bachelor's degree in Computer Science or related field required
• Minimum 5 years of experience in data engineering and distributed computing
• Strong expertise in Apache Spark and performance optimization techniques
• Proficiency in SQL and experience with data warehousing and query tuning
• Familiarity with Python or Scala programming languages
• Working knowledge of cloud ecosystems (AWS, Azure, or GCP)
• Excellent communication and customer-facing skills
• Ability to manage scope, timelines, and deliverables in technical projects
• Experience with machine learning and data science concepts
• Familiarity with CI/CD pipelines and MLOps practices
• Knowledge of Databricks platform and big data architecture design