NDA Partners is seeking a Technical Solutions Engineer to lead the implementation, optimization, and ongoing support of healthcare IT solutions for clients. This is a client-facing technical role that blends hands-on data analysis (SQL, Python, Unix) with healthcare domain knowledge (HL7, FHIR, HIPAA) and strong communication. You’ll be the primary technical point of contact for client implementations — gathering requirements, configuring and deploying solutions, training end users, and resolving escalations.

Job Description

We are seeking an experienced Data Engineer with strong hands-on expertise in Databricks, PySpark, Python, and SQL. The role is responsible for designing, developing, and optimizing scalable data pipelines and data-processing solutions in a cloud-based data environment.

This is a hands-on position. The ideal candidate has personally built and supported Databricks solutions, not just theoretical exposure, and can walk through real examples of PySpark transformations, ETL/ELT pipelines, Delta Lake implementations, performance optimization, data-quality controls, and production troubleshooting.

Responsibilities

Design and develop scalable ETL/ELT data pipelines using Databricks
Develop data-processing and transformation solutions using PySpark and Python
Write complex SQL queries for data transformation, validation, and analysis
Build and maintain solutions using Delta Lake and Lakehouse architecture
Integrate data from multiple sources, including databases, APIs, files, and cloud storage
Optimize Spark jobs, Databricks workloads, and SQL queries for performance and cost efficiency
Implement data-quality, validation, reconciliation, monitoring, and error-handling mechanisms
Troubleshoot production data issues and perform root-cause analysis
Collaborate with business analysts, architects, and application teams to translate business requirements into technical solutions
Follow engineering best practices for testing, source control, CI/CD, documentation, and production deployment

Required Skills & Experience

5–8 years of overall Data Engineering experience
Strong hands-on experience with Databricks
Strong hands-on development experience with PySpark / Apache Spark
Strong Python programming skills and advanced SQL skills
Experience developing production-grade ETL/ELT pipelines
Experience with Delta Lake and Lakehouse concepts
Experience processing large-volume and complex datasets
Solid understanding of data modeling, data quality, and data governance principles
Strong troubleshooting and performance-tuning skills
Experience with at least one cloud platform: Microsoft Azure, AWS, or Google Cloud
Experience with Git and CI/CD practices

Nice to Have

Databricks Unity Catalog experience
Databricks Workflows / Jobs and cluster optimization experience
Experience with Azure Data Factory or Apache Airflow
Experience with Kafka or Spark Structured Streaming
Knowledge of cloud-native storage and data services
Databricks or cloud certification
Experience in banking or financial-services data environments

NDA PARTNERS INC

Open Positions (1)

Data

Data Engineer with Databricks