Key skills
Data EngineeringSQLPySparkDatabricksAirflowData ModelingTeam LeadershipProblem-SolvingCommunication SkillsCollaboration SkillsAIMLETLAnalyticsdbtDjangoElasticsearchLeadershipMentoringCommunicationCollaboration
About this role
YipitData is the leading market research and analytics firm for the disruptive economy, seeking a highly skilled Data Engineering Manager to join their Data Engineering team. This role involves technical leadership in building and scaling data pipelines, ensuring the delivery of reliable and efficient data solutions.
Responsibilities:
- Own the design, build, and optimization of end-to-end data pipelines that power our vendor universe
- Establish and enforce best practices in data modeling, orchestration, and system reliability
- Collaborate with product, engineering, and business stakeholders to translate requirements into robust, scalable data solutions
- Work extensively with Databricks and Airflow for large-scale data processing and orchestration
- Troubleshoot and resolve complex pipeline issues to ensure reliability and performance
- Contribute to the team’s technical strategy, helping drive improvements in scalability, performance, and efficiency
- Lead, mentor, and support engineers through challenges, code reviews, and project execution
Requirements:
- 6+ years of professional experience in Data Engineering or equivalent technical roles (e.g., data architecture, big data development, or ETL engineering)
- 2+ years of managerial experience, including mentoring, team leadership, and supporting delivery
- Strong expertise in SQL and distributed data systems
- Proficiency with PySpark and Databricks for processing and scaling large datasets
- Hands-on experience with Airflow for pipeline orchestration (Dagster/dbt a plus)
- Proven track record of delivering in fast-paced, deadline-driven environments with minimal oversight
- Strong problem-solving skills and ability to translate business needs into scalable technical solutions
- Excellent communication and collaboration skills with both technical and non-technical stakeholders
- Experience leveraging AI/ML models, vector search, or Elasticsearch to enhance data pipelines
- Familiarity with Django or similar web frameworks to support data-driven applications