Coordinate the design, provisioning, and configuration of Databricks workspaces, clusters, and job schedules across development, test, and production environments.
Maintain the platform’s reliability (SLAs/SLOs), capacity planning, cost‑optimization, and performance tuning.
Establish automated monitoring, alerting, and incident‑response processes (Databricks Jobs UI, CloudWatch, Azure Monitor, etc.)
Evaluate and integrate complementary technologies (e.g., data ingestion tools, orchestration, cataloging, governance, MLflow, Spark, Delta Sharing)
Serve as a liaison between data engineering, data science, analytics, and business units.
Establish metrics to evaluate the performance and reliability of Databricks and implement improvements based on findings.
Ensure that the Data Ecosystem is compliant with relevant data regulations and security standards, implementing necessary controls for data security, access control, and compliance.
Requirements
Must have a minimum of 5 years of relevant experience with a Bachelors degree; 3 Years of relevant experience with Masters degree, or 9 years of relevant experience in lieu of the degree requirement
Must have a minimum of 3 years experience working with or supporting Data Ecosystem tools
Must have a minimum of 2 years experience with AWS cloud
Must have exceptional verbal and written communication skills, with the ability to convey complex technical concepts to non-technical stakeholders
Must have strong communication and stakeholder‑management abilities.
Tech Stack
AWS
Azure
Cloud
Spark
Benefits
Health insurance coverage
Life and disability insurance
Savings plan
Company paid holidays
Paid time off (PTO) for vacation and/or personal business