Databricks Data Engineer Architect

Apolis is recruiting for a confidential client seeking a Databricks Data Engineer Architect. The role designs and implements scalable data solutions and leads Salesforce and NetSuite data migrations into Databricks, including architecture, pipeline development, governance, and technical leadership.

Responsibilities:

• Design and architect end-to-end data engineering solutions using Databricks and cloud-based data platforms
• Lead the architecture, design, and implementation of scalable data pipelines for batch and incremental data processing
• Architect and execute data migration solutions from Salesforce and NetSuite to Databricks, ensuring data accuracy, completeness, consistency, and integrity
• Integrate data from Salesforce CRM, NetSuite ERP, and other enterprise source systems into a centralized lakehouse platform
• Develop and optimize ETL/ELT pipelines using PySpark, SQL, Python, and Databricks workflows
• Implement the Databricks Lakehouse architecture, including Delta Lake, data modeling, data transformation, and storage optimization
• Establish data ingestion, transformation, orchestration, monitoring, and error-handling frameworks
• Define and implement data governance, security, access controls, data quality, and metadata management standards
• Optimize data pipelines and Databricks workloads for performance, scalability, reliability, and cost efficiency
• Collaborate with business stakeholders, enterprise architects, data analysts, and engineering teams to understand requirements and deliver technical solutions
• Create architecture diagrams, technical design documents, data mapping specifications, and migration strategies
• Provide technical leadership, conduct code reviews, and mentor data engineering teams
• Ensure adherence to enterprise architecture standards, development best practices, and data compliance requirements

Requirements:

• Extensive hands-on experience with Databricks data engineering and solution architecture
• Proven experience architecting and delivering end-to-end enterprise data solutions using Databricks
• Demonstrated experience migrating data from Salesforce and NetSuite to Databricks
• Strong expertise in Python, PySpark, SQL, and ETL/ELT development
• In-depth knowledge of Delta Lake, Lakehouse architecture, and scalable data processing
• Experience integrating SaaS and enterprise applications with modern cloud data platforms
• Strong understanding of data modeling, data warehousing, dimensional modeling, and data lake design
• Experience designing incremental data loads, change data capture (CDC), and data reconciliation processes
• Hands-on experience with workflow orchestration, pipeline monitoring, troubleshooting, and performance tuning
• Knowledge of cloud platforms such as Microsoft Azure, AWS, or Google Cloud, preferably aligned with the organization's Databricks environment
• Experience with data security, governance, access management, and production deployment practices
• Excellent communication, problem-solving, stakeholder management, and technical documentation skills
• Databricks certifications in data engineering or solution architecture
• Experience with Unity Catalog, Delta Live Tables/Lakeflow, and Databricks Workflows
• Familiarity with Salesforce APIs, Bulk API, and enterprise data extraction patterns
• Familiarity with NetSuite SuiteTalk APIs, SuiteAnalytics, or SuiteQL for data extraction and integration
• Experience with CI/CD, Git, automated testing, and infrastructure-as-code practices
• Experience implementing enterprise data quality frameworks and production-grade data observability