Solution Architect data engineering- Azure Databricks and cloud platform
United States of America
Contract
2 hours ago
$56 - $80 USD
No H1B
Key skills
Azure DatabricksPySparkDelta LakeCloud Architecture - AzureData ArchitectureData Modeling - OLTPData Modeling - OLAPData Modeling - DimensionalLakehouse Design PatternsData Warehouse Design PatternsETL/ELT PipelinesOrchestration FrameworksCI/CDDevOps PracticesAzure DevOpsGitMetadata ManagementData LineageData GovernanceUnity CatalogMicrosoft PurviewAI/ML Data ArchitecturesMulti-cloud EnvironmentsMedallion ArchitectureUS Healthcare Data DomainFHIRHL7EMR Data ModelsPayment IntegrityRisk AdjustmentStakeholder EngagementAIMLMLOpsETLELTData EngineeringAnalyticsSnowflakeDatabricksSparkAzureGCPLeadership
About this role
Yoh, A Day & Zimmermann Company is seeking a Solution Architect specializing in data engineering on Azure Databricks and cloud platforms. The role involves designing enterprise-scale data solutions and ensuring the architecture supports analytics, AI/ML, and operational use cases.
Responsibilities:
- Define and implement end-to-end data architecture (ingestion ? transformation ? consumption)
- Design scalable Lakehouse architectures (Databricks, Delta Lake)
- Develop reference architectures, reusable frameworks, and design patterns
- Align architecture with performance, scalability, security, and cost optimization goals
- Define canonical data models, dimensional schemas, and semantic layers
- Architect enterprise data across domains: Claims, Member, Provider, Eligibility
- Ensure data consistency, interoperability, and reuse across teams
- Architect solutions using: Azure Databricks (Mandatory)
- Guide engineering teams on: Spark optimization, Delta Lake design, and data pipeline best practices
- Enable multi-cloud integrations when required
- Define and enforce: Data governance frameworks (lineage, quality, metadata)
- Access control, RBAC, and data security policies
- Ensure compliance with: HIPAA, HITRUST, and enterprise regulatory standards
- Establish data quality SLAs and auditability mechanisms
- Work closely with: Data Engineering teams, Platform Engineering teams, AI/ML Engineering teams
- Provide architectural oversight for pipelines, workflows, and platform capabilities
- Drive standardization across CI/CD, DataOps, and MLOps practices
- Collaborate with business, product, and analytics teams to: Translate requirements into scalable technical solutions
- Lead architecture reviews, design approvals, and governance forums (ARB)
- Communicate architecture decisions and trade-offs to leadership
Requirements:
- Experience: 10+ Years
- Education: Bachelor's Degree (4-year) – Computer Science / Engineering / Related Field
- Strong experience in Azure Databricks (Mandatory) – PySpark, Delta Lake
- Cloud Architecture (Azure preferred)
- Deep expertise in Data Architecture & Data Modeling (OLTP, OLAP, Dimensional)
- Lakehouse / Data Warehouse design patterns
- Hands-on experience with ETL/ELT pipelines and orchestration frameworks
- CI/CD and DevOps practices (Azure DevOps, Git)
- Metadata, lineage, and governance tools (Unity Catalog, Purview)
- Define and implement end-to-end data architecture (ingestion ? transformation ? consumption)
- Design scalable Lakehouse architectures (Databricks, Delta Lake)
- Develop reference architectures, reusable frameworks, and design patterns
- Align architecture with performance, scalability, security, and cost optimization goals
- Define canonical data models, dimensional schemas, and semantic layers
- Architect enterprise data across domains: Claims, Member, Provider, Eligibility
- Ensure data consistency, interoperability, and reuse across teams
- Architect solutions using Azure Databricks (Mandatory)
- Azure Data Services (ADF, ADLS, Synapse)
- Guide engineering teams on Spark optimization, Delta Lake design, and data pipeline best practices
- Enable multi-cloud integrations when required
- Define and enforce data governance frameworks (lineage, quality, metadata)
- Access control, RBAC, and data security policies
- Ensure compliance with HIPAA, HITRUST, and enterprise regulatory standards
- Establish data quality SLAs and auditability mechanisms
- Work closely with Data Engineering teams, Platform Engineering teams, AI/ML Engineering teams
- Provide architectural oversight for pipelines, workflows, and platform capabilities
- Drive standardization across CI/CD, DataOps, and MLOps practices
- Collaborate with business, product, and analytics teams to translate requirements into scalable technical solutions
- Lead architecture reviews, design approvals, and governance forums (ARB)
- Communicate architecture decisions and trade-offs to leadership
- Experience with AI/ML data architectures (feature stores, model pipelines)
- Multi-cloud environments (Azure + GCP/Snowflake)
- Strong understanding of Medallion architecture (Bronze–Silver–Gold)
- Domain Experience (Good to Have) US Healthcare ecosystem: Claims, Member, Provider datasets
- Familiarity with FHIR / HL7 / EMR data models
- Payment Integrity / Risk Adjustment use cases