Databricks Data Engineer SME - Clearance Required

LMI is a digital solutions provider delivering technology and mission-ready solutions to federal agencies. The Databricks Data Engineer SME will serve as a senior technical lead designing and implementing a governed revenue-cycle data architecture for the Defense Health Agency within Databricks and the War Data Platform. The role also supports production dashboards, operational visualizations, KPI reporting, data quality, governance, orchestration, and collaboration with engineering, analytics, and revenue-cycle teams.

Responsibilities:

• Lead the technical design and implementation of the RevOS data architecture within Databricks and the DHA War Data Platform
• Assess and remediate DHA's existing revenue-cycle data structures, including fragmented schemas, duplicate entities, inconsistent definitions, missing relationships, weak keys, data-quality issues, reconciliation problems, and structures that do not align to the operational workflow
• Develop a target-state canonical revenue-cycle data model connecting beneficiary/patient, encounter, appointment, eligibility, authorization, provider, MTF, documentation, diagnosis, procedure, coding, charge, claim, claim line, payer, adjudication, remittance, payment, adjustment, denial, AR, appeal, follow-up, and recovery opportunity
• Design and implement the Databricks Bronze → Silver → Gold architecture for RevOS
• Design Gold data products and analytical data marts specifically optimized to support Databricks dashboards and visualization workloads
• Develop and maintain Databricks SQL dashboards, AI/BI dashboards, and native visualization capabilities supporting operational, financial, revenue-recovery, and executive users
• Build reusable semantic datasets and optimized SQL structures that allow dashboards to consistently calculate RevOS KPIs across enterprise, DHN, MTF, department, provider, encounter, and claim levels
• Ensure dashboard queries and Gold datasets are optimized for performance, refresh cadence, scalability, and concurrent users
• Configure dashboard-level access consistent with Unity Catalog permissions, row-level security, column-level security, PHI/PII requirements, and user roles
• Support dashboard drill-down from summary Healthy / At Risk / Critical indicators into underlying record-level exceptions and work queues
• Profile Government-furnished Bronze datasets and develop source-to-target mappings
• Normalize and conform data from MHS GENESIS Millennium, Abacus, and other approved sources
• Develop reusable Silver-layer entities with common keys, normalized timestamps, enterprise reference dimensions, and standardized business definitions
• Build Gold products supporting:
• Revenue-cycle KPIs
• Coding-audit financial impact
• Charge completeness
• Claims readiness
• Days-to-Bill
• Denial management
• Payer performance
• Payment/remittance reconciliation
• Underpayment detection
• AR aging
• Revenue recovery
• Audit and NFR traceability
• Design the data model supporting the Revenue Opportunity Ledger and associated recovery work queues
• Establish entity-resolution methodologies connecting records across disparate source systems
• Develop deterministic and, where appropriate, probabilistic matching for encounters, claims, charges, payments, providers, payers, and related entities
• Develop and maintain ODCS-based machine-readable data contracts
• Implement automated quality controls addressing completeness, uniqueness, referential integrity, schema conformity, temporal integrity, code validity, financial reconciliation, and source-to-target consistency
• Establish automated Bronze-to-Silver and Silver-to-Gold quality gates
• Build reconciliation controls across billed, allowed, paid, adjusted, patient-responsibility, AR, and recovery values
• Configure and manage Databricks Unity Catalog, including catalog/schema/table design, classification, tagging, ownership, lineage, row/column security, and access controls
• Support enterprise metadata federation and DHA data-governance requirements
• Develop scalable Delta Lake structures and optimize partitioning, clustering, SQL performance, storage, and compute usage
• Develop Databricks jobs, workflows, orchestration, production monitoring, alerting, and error-handling processes
• Version-control pipeline code, transformations, data contracts, configurations, and infrastructure-as-code using Government-furnished GitLab
• Support automated CI/CD and controlled production promotion
• Partner with Data Scientists and Analytics Engineers to support predictive modeling, payer analytics, recovery scoring, anomaly detection, and dashboard requirements
• Partner with Revenue Cycle and Coding SMEs to translate hospital revenue-cycle workflows into executable data structures and business rules
• Develop source-to-target documentation, data dictionaries, lineage artifacts, architecture diagrams, runbooks, and sustainment documentation
• Mentor Data Engineers and establish reusable Databricks engineering and visualization patterns across the RevOS environment

Requirements:

• Active SECRET Clearance
• Bachelor's Degree and 10+ years of experience in enterprise data engineering, data architecture, data platform development, or related disciplines
• Senior/SME-level hands-on experience with Databricks
• Demonstrated experience using Databricks visualization and dashboard capabilities, including Databricks SQL and/or AI/BI dashboards
• Ability to design the underlying Gold/semantic data structures required for enterprise-scale dashboards and operational reporting
• Experience creating and optimizing SQL queries, datasets, and views that support interactive visualization and drill-down
• Strong experience with:
• Apache Spark / PySpark
• SQL
• Python
• Delta Lake
• ETL / ELT
• Data pipeline orchestration
• Large-scale data transformation
• Demonstrated experience implementing Bronze / Silver / Gold medallion architectures
• Strong experience designing normalized and analytical data models for complex enterprise environments
• Demonstrated ability to evaluate and redesign or remediate poorly structured existing data environments, rather than simply building additional pipelines on top of them
• Experience designing canonical data models, conformed dimensions, enterprise keys, entity resolution, and reusable semantic structures
• Strong knowledge of data-quality engineering, financial reconciliation, metadata, lineage, and data governance
• Experience with Databricks Unity Catalog or equivalent enterprise data-governance/catalog technology
• Experience building production-grade pipelines with automated testing, monitoring, logging, failure handling, and CI/CD
• Ability to translate operational workflows into logical and physical data architectures and visualization-ready products
• Experience supporting highly regulated, healthcare, financial, or Government environments
• Ability to work with PHI, PII, CUI, and other controlled data under applicable security and privacy requirements
• Ability to work across engineering, analytics, visualization, cybersecurity, product, architecture, and business-SME teams
• Ability to meet applicable DHA/DoD security, privacy, access, and data-handling requirements
• Prior experience with Advana and/or the current War Data Platform (WDP)
• Direct experience developing Databricks data products and dashboards in a DoD enterprise environment
• Experience with DHA, MHS, or other DoD healthcare data
• Experience with MHS GENESIS / Oracle Health/Cerner Millennium and/or Abacus
• Healthcare revenue-cycle data experience involving encounters, coding, charge capture, claims, denials, adjudication, remittance, payments, AR, and revenue recovery
• Familiarity with healthcare EDI transactions including 837, 835, 270/271, 276/277, and 278
• Experience implementing Open Data Contract Standard (ODCS) or comparable machine-readable contracts
• Experience with GitLab-based DevSecOps, infrastructure-as-code, automated testing, and secure promotion
• Familiarity with Collibra or comparable enterprise data catalogs
• Experience supporting financial auditability, reconciliation, lineage, and audit-remediation initiatives
• Familiarity with DoD RMF, NIST, IL4/IL5 environments, and federal data-governance requirements