Director, Data Platform Engineering (Databricks)

Live Nation Entertainment is the world’s leading live entertainment and eCommerce company, and they are seeking a Director of Data Platform Engineering to lead efforts in advancing the capabilities of their data platform. The role involves defining strategic direction, overseeing tool development for data ingestion, and collaborating with cross-functional teams to ensure the success of data platform services.

Responsibilities:

• Define and execute a comprehensive strategy for the enhancement and expansion of Data Platform tools and services, aligned with the company's overall goals and market trends
• Lead a team of data engineers and software engineers and collaborate closely with cross-functional teams to ensure the successful delivery of scalable and efficient data ingestion services and common tools
• Build or buy tools to monitor job run status, data quality and SLA thresholds
• Manage cross team dependencies and work with tech leads on solutions
• Encourages a culture of team driven decision making, ownership and accountability
• Develop a solid understanding of Databricks Unified Platform
• Manage and mentor a team of data platform engineers, fostering a culture of collaboration, innovation, and continuous learning
• Set performance expectations, conduct regular performance evaluations, and provide feedback to team members
• Identify skills gaps and training needs within the team and develop appropriate development plans
• Foster a diverse and inclusive work environment, ensuring equal opportunities for all team members
• Oversee the design, development, and maintenance of scalable data engineering infrastructure and efficient data ingestion ensuring optimal performance and reliability
• Drive the implementation of efficient data processing pipelines and workflows to support analytics, reporting, and machine learning initiatives

Requirements:

• Proven experience (10+ years) in data engineering, with a strong emphasis on managing complex API development and data ingestion. Good software engineering experience is required
• Expert in Databricks performance optimization and data ingestion techniques offered by Databricks
• Expert in understanding the new features of Databricks in real world use cases
• Strong experience in Spark, Python, Databricks and ETL is required
• Expert in developing scalable and efficient data pipelines using PySpark
• Highly knowledgeable in selecting right tools and technology stacks for data ingestion
• Strong leadership and management skills, as well as experience leading and developing high-performing teams
• Requires a deep understanding of data engineering principles, strong managerial skills, and the ability to drive innovation and operational excellence
• Deep technical experience across multiple technologies, governance frameworks, data security, and data platforms
• Strong track record partnering with product stakeholders to refine technical data platform roadmap
• Thought leader in organizations defining and implementing data mesh based platforms and ingestion strategies to enable various data needs
• Excellent oral and written communication and presentation skills
• Demonstrated experience with cross-functional teams in a geographically and culturally diverse environment
• Exceptional strategic thinking, problem-solving, and decision-making skills
• Excellent collaboration and partnering skills
• Excellent people management skills and the ability to influence
• Manage multiple priorities, be results-oriented, and be delivery-focused
• Experience using ANSI SQL
• Experience with cloud services from Amazon AWS, Google GCP, or Microsoft Azure is required
• Good experience in full stack development including react UI development and developing backend services for data ingestion