Key skills
AzurePySparkSQLData LakeAnalyticsBIDatabricks
About this role
Role Overview
- Azure Data Engineer with Databricks and PySpark
- Perform Design, Development & Deployment using Azure Services (Data Factory, Databricks, PySpark, SQL)
- Develop and maintain scalable data pipelines
- Build new Data Source integrations to support increasing data volume and complexity.
- Create Technical Specification Design, Application Interface Design.
- Develop batch processing and integration solutions and process Structured and Non-Structured Data
- Collaborate and engage with BI & analytics and the business team
Requirements
- 10 to 13 Years – Experience in Perform Design, Development & Deployment using Azure Services (Data Factory, Databricks, PySpark, SQL)
- Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.
- Experience in creating Technical Specification Design, Application Interface Design.
- Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark
- Develop batch processing and integration solutions and process Structured and Non-Structured Data
- Demonstrated in-depth skills with Azure Databricks and PySpark, and SQL
- Collaborate and engage with BI & analytics and the business team
- Minimum 2 year of Project experience in Azure Databricks
- Minimum 2 years of experience in ADF
- Minimum 2 years of experience in PySpark
Tech Stack
- Azure
- PySpark
- SQL