Remote: Sr / Lead Platform Architect - Azure Databricks

Our Decision Intelligence (DI) team is looking for a Senior / Lead Platform Architect to define and govern the Azure Databricks platform patterns for AI-ready data and RAG across the enterprise. This role will drive data architecture initiatives, guiding the design and implementation of robust, scalable, and secure data solutions. Responsible for utilizing advanced understanding and working knowledge of complex IT business problems, including the ability to design solutions that address business problems, needs, and opportunities. Work across both the Technology organization as well as the Business Stakeholder organizations to ensure the appropriate design, utilization, and governance of Client application landscape as a strategic technology and business asset. Provide technical support and leadership in the creation and delivery of technological solutions designed to meet customers’ business needs. Leverage knowledge, industry experience and best practices to proactively drive definition and evolution of a long-term vision to help provide competitive differentiation, as well as driving corresponding roadmap(s) to accomplish this vision.
This role will also lead the data modeling strategy for the “Intelligent Data Platform,” including the design of conceptual, logical, and physical data models, canonical data structures, and cross-domain information standards that ensure consistent, governed, interoperable data across Client''s platforms and business unit segments.
Keys Roles & Responsibilities:
Platform Architecture Leadership:
· Define and maintain the Azure Databricks reference architecture for AI data preparation, grounding (RAG), orchestration, telemetry, and governance.
· Establish Databricks platform standards and guardrails, including workspace patterns, Unity Catalog design, compute policies, and cost controls.
· Ensure Unity Catalog is the system of record for AI data access, enforcing fine grained permissions, data masking, lineage, and auditability.
· Standardize embedding, feature, context, and underlying data model design to enable reuse of AI-ready data assets across use cases and business domains.
· Partner with business, analytics, and engineering teams to translate business capabilities and processes into scalable data models.
· Architect secure integration patterns between Databricks and downstream AI services or applications, preventing unapproved data egress.
· Embed quality engineering into AI pipelines using MLflow, evaluation datasets, telemetry, and drift monitoring before production rollout.
· Ensure production readiness and operability of Databricks AI workloads through Jobs/Workflows standards, monitoring, and KTLO handoff.
· Apply AI security and compliance by design within Databricks, including identity enforcement, sensitive data protection, and audit logging.
· Govern data model lifecycle management, including metadata standards, lineage, schema evolution, versioning, and model review processes.
· Ensure data models support AI/ML, analytics, and operational use cases while preserving consistency, traceability, and regulatory compliance.
Minimum Requirement:
Degree or equivalent and typically requires 10+ years of relevant experience
Required Qualifications
· 10+ years in platform, solution, or enterprise architecture, including significant hands-on experience in enterprise data architecture and data modeling within Azure-based environments such as Azure Databricks.
· Proven experience designing AI/analytics data platforms, including governance, security, large-scale data access patterns, and data model alignment across multiple domains and source systems.
· Strong understanding of RAG, vector retrieval, data governance, metadata management, lineage, and observability in production environments.
· Experience working in regulated environments with security, privacy, compliance, and data stewardship/model governance requirements.
Preferred Qualifications
· Experience establishing or governing enterprise information models, business glossaries, and semantic layers for analytics and AI.
· Familiarity with data modeling tools and modeling methodologies such as normalized modeling, dimensional modeling, Data Vault, or domain-driven design.
· Familiarity with Unity Catalog, MLflow, Vector Search, and Azure-native security patterns.
· Experience integrating enterprise data models with metadata, catalog, lineage, and governance tooling.
· Azure certifications (AZ 305, AI 102, AZ 500) strongly preferred.