Lead the design and implementation of scalable data platforms and pipelines using PySpark, Spark, and Python
Drive adoption and best practices for Palantir Foundry (data pipelines, ontology, workflows, and operational applications)
Architect and optimize high-performance data processing solutions for large-scale datasets
Provide technical leadership and mentorship to data engineers, ensuring code quality and best practices
Collaborate with cross-functional teams (business, analytics, data science) to translate requirements into scalable solutions
Design robust data models, ETL/ELT frameworks, and data integration strategies
Ensure data quality, governance, security, and compliance across enterprise data platforms
Lead performance tuning, troubleshooting, and optimization of data pipelines
Drive CI/CD implementation, code reviews, and release management
Stay current with emerging data engineering technologies and recommend improvements

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
8+ years of experience in data engineering, with at least 2–3 years in a technical leadership role
Strong hands-on experience with: Python & PySpark, Apache Spark, Advanced SQL, Experience with Palantir Foundry or similar modern data platforms, Deep understanding of data engineering principles (ETL/ELT, data modeling, distributed systems)
Experience designing and managing large-scale data architectures
Strong leadership, communication, and stakeholder management skills

Data Engineer, Palantir, Spark, PySpark, Python

Key skills