Job Title: Databricks Architect / Resident Solution Architect
Openings: 2
Location: San Francisco, CA – Local preferred; Remote considered
Work Hours: Must be able to work PST hours
Duration: 8–12 Months
Visa: Any
Rate: flexible
Industry: Banking / Financial Services
Experience: 12–15+ Years
We are looking for a highly experienced Databricks Architect / Resident Solution Architect with strong hands-on Data Engineering and Databricks architecture experience.
The ideal candidate should have extensive experience designing and delivering enterprise-scale Databricks Lakehouse solutions and must have prior Banking/Financial Services domain experience.
Architect and deliver enterprise-scale solutions using the Databricks Lakehouse Platform.
Lead end-to-end Databricks implementations from architecture and design through production deployment.
Design scalable, secure, high-performance, and cost-efficient data platforms.
Develop and optimize ETL/ELT pipelines using Databricks, Apache Spark, Delta Lake, and Structured Streaming.
Design modern data warehouse and Lakehouse architecture solutions.
Implement data governance and security using Unity Catalog.
Optimize Spark workloads, including partitioning, execution plans, memory management, and runtime performance.
Design cloud-native data solutions across AWS, Azure, and/or Google Cloud Platform.
Implement CI/CD and DevOps practices for enterprise data platforms.
Work with MLflow and MLOps practices for machine learning lifecycle management, model deployment, and monitoring.
Troubleshoot complex Databricks, Spark, and distributed data-processing issues.
Provide architecture recommendations based on business and technical requirements.
Work directly with client stakeholders and provide technical leadership throughout implementation.
12–15+ years of experience in Data Engineering, Data Platforms, Data Analytics, and Modern Data Warehouse solutions.
10+ years of consulting and client-facing delivery experience.
Banking/Financial Services experience is mandatory.
Experience delivering 6–8+ end-to-end Databricks implementations as a hands-on developer, technical lead, or solution architect.
Advanced hands-on experience with Databricks and Apache Spark.
Strong experience with Delta Lake, Structured Streaming, Unity Catalog, Workflows, MLflow, and Delta Live Tables (DLT).
Strong Spark performance tuning and optimization experience.
Deep hands-on experience with at least one major cloud platform: AWS, Azure, or Google Cloud Platform.
Experience developing large-scale ETL/ELT data pipelines.
Experience with DevOps and CI/CD using Azure DevOps, GitHub Actions, GitLab CI/CD, or Jenkins.
Working knowledge of MLOps, model deployment, lifecycle management, and monitoring.
Strong understanding of distributed data processing and scalable data architecture.
Strong client-facing communication and consulting skills.
Required/Strongly Preferred: Databricks Data Engineer Professional certification or equivalent advanced Databricks certification.
Good to Have: Databricks Solutions Architect Champion recognition/program experience.
Please share your updated resume along with your current visa/work authorization status at for further consideration.
