Key skills
DatabricksUnity CatalogData GovernanceAccess ControlData QualityData LineageData CatalogingMetadata ManagementBusiness SemanticsSQLPythonData Pipeline DevelopmentDelta LakeAIMLLLMCommunication
About this role
Lumenalta is a technology solutions company that partners with organizations to accelerate business growth through innovative technology. They are seeking experienced Data Platform Engineers with expertise in Databricks and Unity Catalog to build scalable data foundations and ensure data governance across their platforms.
Responsibilities:
- Own data governance within Databricks, primarily through Unity Catalog, across access control, cataloging, auditing, lineage, data quality, business semantics, cost controls, and data sharing
- Design and implement access control and security models (catalogs, schemas, row/column-level permissions) within Unity Catalog
- Build and maintain data quality frameworks and validation rules directly in the platform (e.g., Delta Live Tables expectations, constraints)
- Certify tables and identify "gold standard" datasets, establishing what's official and production-ready across the lakehouse
- Define and maintain business semantics (metadata, glossaries, table/column documentation) that ground AI and natural-language query systems in accurate context — critical for preventing hallucinations in downstream AI use cases
- Implement lineage tracking and auditing across pipelines and consumption layers
- Manage cost controls and governance over compute/storage usage within Databricks
- Partner with data engineers, AI/ML teams, and business stakeholders to ensure governance is embedded across the full platform, not siloed
- Stay current on Unity Catalog capabilities across data, ML, and AI workloads
Requirements:
- Strong, hands-on experience with Databricks and Unity Catalog — this is a platform-native governance role, not a legacy MDM/DQ tool role
- Demonstrated experience building governance frameworks: access control, data quality, lineage, cataloging, and metadata/business semantics
- Comfort working across the full data platform — pipelines, ML, and AI workloads — with enough AI literacy to support semantic layers that power LLM/AI applications
- Proficiency in SQL, Python, and data pipeline development
- Excellent communication skills — able to work cross-functionally as governance touches every part of the project, not as a standalone function
- Bachelor's or Master's degree in Computer Science, Information Systems, or related field
- Experience with Delta Lake architecture preferred