CAQH is the trusted data connector at the core of healthcare, and they are seeking a Data Engineer to design, develop, and optimize advanced data pipelines and infrastructure. This role is critical for delivering scalable data solutions that support analytics and AI/ML initiatives across the organization.
Responsibilities:
- Design, build, and maintain complex ETL/ELT pipelines across Databricks, Azure SQL, and downstream gold-layer models supporting priority projects
- Lead development and evolution of enriched data models, including field-level enrichment logic, recency rules, and provider-level enrichment flags
- Own data logic, including reconciliation between source and target data sources and resolution of duplication and data discrepancies
- Implement and refine medallion architecture patterns (bronze → gold), ensuring data quality, traceability, and performance at scale
- Identify, document, and remediate systemic data quality issues, including null handling, soft deletes, authorization flags, and incorrect organizational mappings
- Define and operationalize rules for data in collaboration with product, governance, and engineering stakeholders
- Produce authoritative documentation (Confluence, mapping workbooks) to serve as a single source of truth for enrichment logic and data behavior
- Act as a primary technical counterpart for vendors providing detailed queries, validation logic, and corrective guidance on upstream data issues
- Partner closely with product owners, architects, and application teams to ensure data models align with product defined use cases
- Support UAT and release readiness by preparing data, validating counts, and resolving last‑mile data defects under tight timelines
- Provide hands-on technical leadership without formal direct reports, mentoring peers and guiding best practices in SQL, Databricks notebooks, and data modeling
- Influence architectural decisions related to Databricks compute, job scheduling, and environment promotion strategies
Requirements:
- 4–7 years of hands-on experience in a data engineering or analytics engineering role
- Demonstrated success leading data modernization or migration initiatives in cloud environments
- Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related field
- Advanced SQL expertise (complex joins, reconciliation, performance tuning)
- Deep hands-on experience with Databricks, Delta Lake, and Azure SQL
- Strong data modeling skills for analytical, operational, and API‑driven use cases
- Proven ability to debug and stabilize messy, evolving enterprise data domains
- Excellent written and verbal communication, especially for explaining complex data behavior to non‑technical stakeholders
- Experience using Git, DevOps tools, and CI/CD pipelines for data engineering workflows
- Azure Data Engineer Associate or related certification
- Coursework or certification in AI/ML