Databricks Administrator

Role Overview

• Administer, configure, and support the Databricks platform to enable scalable data engineering, analytics, and AI/ML workloads for the enterprise.
• Ensure platform reliability, security, cost optimization, and compliance with relevant standards and organizational policies.
• Provide technical support, guidance, and troubleshooting assistance to data engineers, analysts, and data scientists across various business units.
• Manage workspace configuration, cluster policies, and user access controls to maintain secure and efficient operation of the Databricks environment.
• Integrate Databricks with cloud services and related enterprise platforms to enable seamless data and analytic workflows.
• Proactively monitor platform performance, usage, and system health, implementing corrective actions and improvements as necessary.
• Collaborate with stakeholders to understand business objectives, operational challenges, and user requirements related to data analytics and platform utilization.
• Identify and evaluate alternative solutions, conducting studies and cost/benefit analyses to determine the best approach for organizational needs.
• Analyze user requirements, procedures, and problems to automate processing or improve existing computer systems.
• Engage with personnel from multiple organizational units to assess current operational procedures, identify system deficiencies, and gather input/output requirements such as data input forms and reporting formats.
• Document detailed user requirements, program functions, and necessary steps for the development or modification of computer programs supporting the Databricks platform.
• Review computer system capabilities, specifications, and scheduling limitations to assess the feasibility of requested program changes within existing system constraints.

Requirements

• 8 years of experience administering Databricks workspaces in a cloud environment, specifically AWS.
• 8 years of strong understanding of Databricks cluster configuration, job scheduling, and workspace management.
• 8 years of experience managing user access, roles, and permissions using IAM, SCIM, and role-based access control (RBAC).
• 8 years of proficiency with Apache Spark concepts, including performance tuning and troubleshooting.
• 8 years of experience integrating Databricks with cloud storage services (e.g., S3).
• 8 years of experience implementing and enforcing cluster policies and workspace governance standards.
• 8 years of familiarity with Databricks SQL, notebooks, and job orchestration.
• 8 years of experience monitoring platform health, performance, and availability.
• 8 years of understanding of data security, encryption, and compliance requirements.
• 8 years of experience with DevOps or automation tools such as Terraform, CI/CD pipelines, and scripting.
• Must be able to pass a background check.

Tech Stack

• Apache
• AWS
• Cloud
• Spark
• SQL
• Terraform

Benefits

• Medical, Dental and Vision Insurance
• Wellness Program
• Flexible Spending Accounts (Healthcare, Dependent Care, Commuter)
• Short-Term and Long-Term Disability options
• Basic Life and AD&D Insurance (Company Provided)
• Voluntary Life and AD&D options
• 401(k) Retirement Savings Plan with matching after one year
• Paid Time Off