Oversee the operational health of the platform.
Maintain and enhance monitoring, logging, and alerting solutions to proactively identify and resolve issues.
Lead incident response and perform root cause analysis for platform-wide problems.
Maintain and enhance the core components of our Azure Databricks environment.
Define the strategic roadmap for the platform, including the adoption of new features and best practices.
Manage the enforcement and enhancement of data security and governance policies.
Implement strategies for cost management and performance tuning at a platform level.
Maintain and enhance automated solutions for provisioning and managing Databricks workspaces.
Serve as a subject matter expert and mentor to other data engineers.
Partner with data engineering and business teams to understand their needs.
Oversee and direct the efforts of 2 platform engineers.

8+ years of experience in a hands-on engineering role with a focus on cloud data platforms, distributed systems, and data engineering.
Deep, hands-on experience with the Azure Databricks platform, including its administration, cluster management, job orchestration, and key features like Delta Lake and Unity Catalog.
Expert-level knowledge of the Azure ecosystem, including Azure Data Lake Storage (ADLS), Azure Active Directory (AAD), and networking components.
Strong proficiency in Terraform, Python, Apache Spark/PySpark, SQL, and Shell scripting.
Proven experience with Infrastructure as Code tools (e.g., Terraform) and building automated CI/CD pipelines for platform and data application deployments.
A strong understanding of distributed computing principles and the architecture of large-scale data systems.
Exceptional analytical and problem-solving skills with a track record of diagnosing and resolving complex, platform-level issues.
Excellent communication, documentation, and presentation skills, with the ability to influence technical and business stakeholders.

Staff Data Engineer

Key skills