Senior Data Engineer — Databricks Certified

AbarVa is building Nexus, an AI-native enterprise execution platform alongside complex enterprise Data and AI transformations. The company is seeking a Senior Data Engineer to design, build, debug, and productionize Databricks-based data foundations, pipelines, governed data products, and data-quality controls for analytics and AI.

Responsibilities:

• Build and operate enterprise-scale data pipelines on Databricks
• Develop ingestion patterns for structured and semi-structured enterprise sources
• Build Bronze, Silver and curated data products using modern lakehouse patterns
• Develop with Python, PySpark, Spark SQL and SQL
• Implement metadata-driven ingestion and reusable engineering patterns
• Profile complex enterprise data and identify schema, quality and reconciliation issues
• Build data-quality rules, reconciliation controls and exception-management processes
• Implement lineage, metadata, access and governance patterns using Unity Catalog
• Work with identity, reference-data and cross-system matching problems
• Support reconstruction and modernization of legacy analytics and business logic
• Build automated tests, monitoring, replay, observability and production controls
• Participate actively in code reviews and architecture discussions
• Work closely with analytics, AI/ML and visualization engineers
• Work directly with enterprise client data and technology teams

Requirements:

• Current Databricks certification required
• Databricks Certified Data Engineer Associate or Professional
• Strong hands-on Databricks engineering experience
• Strong Python / PySpark and SQL
• Production experience with Spark and Delta Lake
• Experience with ingestion, schema evolution, incremental processing and data pipelines
• Experience with Bronze/Silver/Gold or comparable layered data architectures
• Experience implementing data quality and reconciliation
• Working knowledge of Unity Catalog
• Ability to troubleshoot real production data-engineering problems independently
• Strong written and spoken English
• Comfortable working directly with enterprise clients and senior engineers
• Be currently authorized to work in the United States
• Be able to work in the United States without current or future employment sponsorship
• Be based in the United States and able to work directly with U.S.-based client teams
• Strong AWS experience preferred; Azure experience also valuable
• Databricks Data Engineer Professional certification
• Healthcare, financial services, insurance or another regulated industry
• Claims, clinical, financial or other complex enterprise data
• CDC and streaming architectures
• Metadata-driven engineering frameworks
• CI/CD and automated testing for data pipelines
• Data governance and Collibra exposure
• Supporting data foundations for AI/ML or agentic applications