Key skills
AzureCloudSQLUnityVaultData EngineeringAnalyticsSnowflakeDatabricksGitHub ActionsAzure DevOpsAzure SQLIdentity ManagementGitHubAgileCI/CD
About this role
Role Overview
- Manage and maintain all Databricks workspaces across development, test, and production environments.
- Administer Unity Catalog / Hive Metastore, including schema design, permissions, external locations, mounts, and data governance configurations.
- Manage clusters, cluster policies, job clusters, cluster performance, auto-scaling, and cost optimization.
- Oversee Databricks Jobs, workflows, task orchestration, alerting, and SLAs.
- Configure and maintain external data sources and connectors (ADLS Gen2, Delta tables, SQL MI, Snowflake, APIs).
- Implement and maintain CI/CD pipelines for Databricks notebooks, SQL, and infrastructure using Azure DevOps/GitHub Actions.
- Develop, maintain, and enforce platform security standards including RBAC, service principals, secrets, key vault integration, and audit logging.
- Administer Azure SQL Managed Instance and other SQL environments supporting the CAQH EDW.
- Perform database performance tuning, indexing, partitioning, storage management, and query optimization.
- Oversee backup/restore operations, failover strategies, and high availability configurations.
- Manage schema changes, deployments, and environment synchronization across dev, test, and prod.
- Partner with the Data Engineering team on efficient ingestion and transformation patterns to SQL-based systems.
- Monitor capacity, costs, and system health for all SQL and storage resources in Azure.
- Administer Azure Data Factory including linked services, integration runtimes, pipelines, triggers, and parameterized workflows.
- Implement enterprise standards for logging, error handling, retries, alerting, and operational monitoring.
- Ensure secure configuration and compliance for all data movement operations, including PII/PHI safeguards.
- Establish and enforce standards for data access, identity management, secrets rotation, key management, and auditability.
- Participate in Data Governance initiatives including metadata management, lineage tracking, and catalog standardization.
- Build documentation for platform configurations, processes, and operational playbooks.
- Evaluate new Azure features and provide recommendations for modernization and cost efficiency.
- Monitor platform reliability, SLAs, performance, and uptime across all Azure data services.
- Troubleshoot and resolve platform-wide issues including job failures, performance bottlenecks, and connectivity disruptions.
- Partner with InfoSec to ensure compliance with CAQH security policies and healthcare data regulations, including but not limited to HITRUST security access reviews.
- Support agile development teams by managing environments, access, and deployment pipelines.
Requirements
- 5–8 years of experience as an Azure Platform Administrator, Cloud Engineer, or Database Administrator
- Hands-on experience managing Databricks environments at scale
- Experience supporting EDW or analytics platforms in Azure or hybrid environments
- Experience building and managing CI/CD pipelines for data workloads
- Healthcare or regulated data experience strongly preferred
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent experience
- Azure Administrator Associate, Azure Database Administrator, or Databricks Administrator certifications preferred
Tech Stack
- Azure
- Cloud
- SQL
- Unity
- Vault
Benefits
- medical, dental, and vision coverage
- a 401(k) with company contributions and matching
- paid parental leave
- tuition assistance
- generous paid time off