Senior Data Engineer - Remote Opportunity!

KinderCare Learning Companies is an early childhood education and child care provider offering programs for children and families across the United States. The Senior Data Engineer will design, optimize, and govern the company’s Databricks-based lakehouse platform, including medallion architecture, Unity Catalog, and enterprise data pipelines. The role also enables BI and AI/ML products, supports data governance and reliability, and mentors data engineering teams.

Responsibilities:

• Design, build, and optimize production-grade ETL/ELT pipelines across the medallion architecture using Delta Lake, Delta Live Tables / Lakeflow Declarative Pipelines, and Databricks Workflows
• Own performance tuning and cost efficiency across the platform — cluster/job sizing, Photon, partitioning and Z-ordering, Liquid Clustering, Auto Loader, and DBU cost governance
• Architect and enforce data models that support enterprise BI (Microsoft Fabric/Power BI) and downstream analytics products
• Build and maintain CI/CD pipelines for Databricks assets (Databricks Asset Bundles, Repos, Git-based deployment) following Agile and Test-Driven Development practices
• Integrate platform pipelines with middleware and source systems (e.g., Boomi) and cloud-native services
• Administer and evolve Unity Catalog: catalogs/schemas, fine-grained access control, lineage, row/column-level security, and workspace-catalog bindings
• Implement and enforce data quality, observability, and reliability practices (expectations/constraints, monitoring, alerting, SLA management) across pipelines
• Partner with security, compliance, and audit teams to maintain SOX ITGC alignment; access reviews, change control, and auditable engineering practices
• Troubleshoot and resolve complex production data pipeline issues, performing root-cause analysis and implementing preventive fixes
• Create clear, durable documentation of architecture, procedures, and operational runbooks
• Evaluate, pilot, and productionize Databricks-native AI/ML capabilities — including Genie for natural-language data access and MLflow for experiment tracking and model lifecycle management, Feature Store, and Model Serving
• Support the build-out of vector search and retrieval-augmented generation (RAG) patterns on top of governed Unity Catalog data for internal AI use cases
• Collaborate with data science and analytics stakeholders to prepare curated, ML-ready Gold-layer datasets and feature pipelines
• Stay current on the Databricks roadmap (Lakehouse AI, Mosaic AI, Agent frameworks) and recommend adoption where it advances platform maturity and business value
• Help define guardrails and human-in-the-loop controls for AI-assisted and agentic engineering workflows introduced to the platform
• Serve as a technical mentor to mid-level data engineers, raising the bar on Databricks best practices, code quality, and architectural rigor
• Partner with the Data Architect and Data Engineering Lead to translate business requirements into technical specifications for BI and AI products
• Collaborate cross-functionally with technical and non-technical stakeholders, including product, security, and business teams
• Contribute to data governance, data security, and data privacy standards across the platform

Requirements:

• Bachelor's degree in computer science, information systems, engineering, statistics, or related field or equivalent work experience
• 7+ years of experience as a data engineer with 3+ years specifically architecting and operating production workloads on Databricks
• Strong understanding of data governance, data security, and access control best practices
• Experience with Agile development methodologies, CI/CD automation, and Test-Driven Development
• Excellent problem-solving skills and the ability to lead technical troubleshooting independently
• Strong written and verbal communication skills with the ability to explain technical concepts to non-technical stakeholders
• Demonstrated experience owning a lakehouse/medallion architecture at scale, including data modeling for BI consumption
• Experience operating in a governed or regulated environment (SOX, HIPAA, or similar) with formal change control and access governance
• Deep, hands-on expertise with the Databricks Lakehouse Platform: Delta Lake, Unity Catalog, Delta Live Tables / Lakeflow, Workflows, and cluster/job optimization (Photon, Auto Loader, Liquid Clustering)
• Advanced SQL and strong Python (PySpark) development skills; comfort with Scala a plus
• Experience with Databricks Asset Bundles, Repos, and CI/CD for lakehouse deployments
• Working knowledge of cloud data services (Azure preferred — ADLS, Azure SQL, Synapse/Fabric; AWS/GCP equivalents acceptable) and cloud migration patterns
• Hands-on experience with MLflow for experiment tracking, model registry, and lifecycle management
• Working knowledge of Databricks AI/ML capabilities — Feature Store, Model Serving, Genie, Mosaic AI, or equivalent lakehouse ML tooling
• Exposure to vector search, embeddings, or RAG architectures and an understanding of how governed data feeds AI/ML products
• Comfort partnering with data science teams on ML-ready data pipelines, even without a formal data science background
• Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Machine Learning Associate/Professional) strongly preferred
• Familiarity with BI integration layers such as Microsoft Fabric/Power BI and enterprise middleware (e.g., Boomi) a plus