Senior Data Platform Engineer II (Databricks)

Aledade, Inc. is seeking a Senior Data Platform Engineer II to architect and manage high-performance, distributed data environments for healthcare analytics. The role involves ensuring the scalability and security of Databricks Lakehouse and Snowflake environments while mentoring junior engineers and collaborating with various stakeholders.

Responsibilities:

• Develop and implement scalable and performant solutions
• Partner, as a peer, with Engineering Managers, Product Managers, and stakeholders throughout Aledade to develop and execute technical roadmaps using Agile processes
• Mentor and coach more junior engineers including thorough pull request reviews for other developers and be receptive to critical feedback on your own work

Requirements:

• BS/BTech (or higher) in Computer Science, Engineering or a related field or equivalent experience
• 6+ years experience as an engineer building and optimizing highly scalable distributed data systems (e.g., Databricks, Spark, or Snowflake)
• 3+ years of experience working with SQL and data modeling on large multi-table data sets
• 3+ years of experience acting as a trusted technical decision-maker in a team setting, solving for short-term and long-term business value
• 3+ years of experience coaching other engineers
• Deep expertise in managing Databricks workspaces, including Unity Catalog for data governance, lineage, and fine-grained access control
• Advanced proficiency with Terraform (or similar) to automate the provisioning and scaling of Databricks clusters, cloud resources (AWS preferred), and networking
• Experience managing Snowflake environments, specifically focusing on warehouse cost optimization, security integration, and secure data sharing
• In-depth knowledge of distributed systems, including partitioning, liquid clustering/Z-Ordering, sharding, and high-availability strategies for petabyte-scale data
• Proven track record in performance monitoring and query tuning for distributed workloads to ensure system reliability and cost-efficiency
• Experience designing and optimizing high-throughput ETL/ELT pipelines and ingestion systems (batch and streaming) using Spark
• Experience building robust CI/CD pipelines for data infrastructure and deploying services using containerization (Docker, Kubernetes)
• Expertise in building systems that handle protected information, with specific experience in HIPAA and SOX compliance frameworks
• Experience navigating health-tech data complexities, such as Electronic Health Records (EHR), clinical data formats (HL7/FHIR), and claims data
• Sitting for prolonged periods of time
• Extensive use of computers and keyboard
• Occasional walking and lifting may be required