Role Overview

Architect and design enterprise-scale data platforms leveraging Azure Databricks and Azure cloud services;
Define and implement Lakehouse architecture patterns using Delta Lake and Databricks best practices;
Design scalable data ingestion, transformation, and serving frameworks supporting both batch and streaming workloads;
Design and implement real-time and near real-time data processing solutions using Structured Streaming, CDC, Change Data Feed (CDF), and event-driven architectures;
Design and manage multi-workspace Databricks environments and shared Metastore architectures.
Establish platform governance, security controls, cluster policies, RBAC models, and access management frameworks;
Design and implement CI/CD pipelines for Databricks assets, infrastructure, and data workloads.
Automate infrastructure provisioning using Infrastructure as Code (Terraform, ARM, or Bicep);
Develop deployment strategies and release management processes across development, staging, and production environments;
Optimize Databricks workspaces, clusters, jobs, and storage configurations for performance and cost efficiency;
Implement monitoring, alerting, logging, and observability solutions for platform operations;
Define and enforce data architecture standards, data quality frameworks, and governance policies;
Integrate Azure Databricks with Azure Data Factory, Azure Synapse, Azure Storage, Azure Key Vault, Azure Entra ID, and other Azure services;
Support Lakeflow Connect, managed connectors, CDC implementations, and enterprise data ecosystem interoperability;
Collaborate with engineering and business teams to translate requirements into scalable technical solutions;
Provide architectural guidance, technical leadership, and mentorship to engineering teams.

Requirements

7+ years of experience in Data Engineering, Data Platform Engineering, Cloud Data Architecture, or Data Platform Consulting.
3+ years of hands-on experience architecting solutions with Azure Databricks.
Strong expertise in Apache Spark, PySpark, Delta Lake, and Databricks Lakehouse architecture.
Hands-on experience with Unity Catalog, data governance, lineage, permissions models, and catalog isolation strategies.
Experience designing and managing Databricks Metastore and multi-workspace architectures.
Strong knowledge of Databricks administration, user provisioning, RBAC, workspace management, and security controls.
Extensive experience designing cloud-native solutions on Microsoft Azure.
Strong knowledge of Azure Data Factory, Azure Storage, Azure Key Vault, Azure Monitor, Azure Entra ID, and Azure networking concepts.
Experience designing streaming and low-latency data pipelines using Databricks Structured Streaming.
Hands-on experience with CDC, Change Data Feed (CDF), and modern data ingestion patterns.
Hands-on experience building and maintaining CI/CD pipelines using Azure DevOps and/or GitHub Actions.
Experience implementing Infrastructure as Code using Terraform, ARM Templates, or Bicep.
Strong understanding of DevOps principles, deployment automation, and environment management.
Strong SQL and Python programming skills.
Excellent communication, stakeholder management, and consulting skills.

Tech Stack

Apache
Azure
Cloud
PySpark
Python
Spark
SQL
Terraform
Unity
Vault

Benefits

Digital-First Approach: Great talent knows no borders! You can work from wherever you are — we hire and collaborate with professionals worldwide.
Remote Work Model: Balance your professional and personal life with our flexible working conditions, empowering you to deliver your best from anywhere.
Exciting Projects: Dive into impactful projects across industries that challenge and spark creativity.
Boost Your Expertise: Grow your career with continuous learning, development opportunities, and hands-on experience.
Join the Best Team Ever: Collaborate with our diverse and cross-cultural team of passionate technologists and creative thinkers.

opinov8

Open Positions (1)

Engineering

Azure Databricks Architect