Key skills
PowerShellAIMLData EngineeringData WarehousingData LakeDatabricksSparkApache SparkAirflowApache AirflowAWSAzureGCPJenkinsAzure DevOpsCI/CDLeadership
About this role
Nasscomm is seeking a Databricks Engineer with extensive experience in Data Engineering. The role involves designing and maintaining scalable data pipelines and cloud-native storage solutions using Databricks and various cloud platforms.
Responsibilities:
- 5+ years of hands-on experience in Data Engineering with strong expertise in Databricks on AWS, Azure, or GCP cloud platforms
- Strong knowledge of Lakehouse architecture, Apache Spark, Delta Lake, PySpark, and enterprise data lake solutions
- Experience designing and maintaining scalable data pipelines, distributed compute platforms, and cloud-native storage solutions
- Hands-on expertise with Databricks tools including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, and Apache Airflow
- Skilled in data warehousing concepts including 3NF, dimensional modeling, metadata-driven ingestion, and data quality frameworks
- Experience implementing Unity Catalog, fine-grained security, governance, compliance, and access control strategies
- Strong understanding of CI/CD pipelines using Azure DevOps, Jenkins, AWS Code Pipeline, TFS, or PowerShell automation
- Proven experience in performance tuning and optimization of Spark/Databricks pipelines, code, and compute resources
- Leadership experience managing cross-functional teams and enterprise-scale data projects
- Exposure to Databricks Lakeflow and AI/ML technologies is a plus
Requirements:
- 5+ years of hands-on experience in Data Engineering with strong expertise in Databricks on AWS, Azure, or GCP cloud platforms
- Strong knowledge of Lakehouse architecture, Apache Spark, Delta Lake, PySpark, and enterprise data lake solutions
- Experience designing and maintaining scalable data pipelines, distributed compute platforms, and cloud-native storage solutions
- Hands-on expertise with Databricks tools including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, and Apache Airflow
- Skilled in data warehousing concepts including 3NF, dimensional modeling, metadata-driven ingestion, and data quality frameworks
- Experience implementing Unity Catalog, fine-grained security, governance, compliance, and access control strategies
- Strong understanding of CI/CD pipelines using Azure DevOps, Jenkins, AWS Code Pipeline, TFS, or PowerShell automation
- Proven experience in performance tuning and optimization of Spark/Databricks pipelines, code, and compute resources
- Leadership experience managing cross-functional teams and enterprise-scale data projects
- Exposure to Databricks Lakeflow and AI/ML technologies is a plus