Databricks Architect

Own end-to-end architecture for the Databricks-on-Azure lakehouse platform, including workspace design, Unity Catalog governance, storage topology, and cluster policies

Define and scalemedallion architecture standards (Bronze, Silver, Gold) across domains

Establishplatform standards and reusable patterns to accelerate Databricks adoption across teams

Design and implement secure, compliant data access frameworks (RBAC, ABAC, row/column-level security)

Lead CI/CD and DevOps patterns for Databricks deployments, ensuring audit-ready pipelines

Optimize performance and cost across Delta Lake pipelines (partitioning, clustering, Photon, optimization strategies)

Build reusable ingestion frameworks for structured and streaming data (API, Kafka, CDC, batch pipelines)

Partner with Data Governance, Security, SRE, and Legal teams to align platform design with healthcare compliance requirements

Provide architectural guidance on PHI/PII data handling and lifecycle management

Evaluate and recommend platform tooling (build vs. buy decisions)

Mentor engineers and architects; establish design reviews and documentation practices

Bachelor's degree in Computer Science, Information Systems, or a related field, or relative experience, and typically requires 7+ years of experience in data engineering and data architecture

Proven experience building and scaling Databricks platforms from the ground up, including workspace provisioning, Unity Catalog, and governance frameworks

Strong hands-on expertise in Databricks and Azure (ADLS Gen2, networking, identity, security)

Demonstrated ability to remain technically hands-on while leading architecture decisions

Deep expertise in Delta Lake, including performance optimization and data lifecycle management

Experience designing enterprise data platforms in regulated environments (e.g., PHI/PII handling)

Strong understanding of data governance, access controls, and compliance frameworks

Proven ability to establish standards, drive adoption, and influence multiple engineering teams

Experience scaling early-stage Databricks platforms and building enterprise-wide standards

Familiarity with streaming and event-driven architectures (Kafka, Structured Streaming)

Experience with Infrastructure-as-Code tools (Terraform, Azure Bicep)

Knowledge of Databricks advanced features (Delta Live Tables, Unity Catalog federation, Delta Sharing)

Experience supporting AI/ML data readiness and feature engineering pipelines