Key skills
AWSETLPythonUnityData EngineeringData LakeDatabricksLambdaS3IAMGluePerformance Optimization
About this role
Role Overview
- Design, develop, and maintain scalable data pipelines using Databricks and Python
- Build and optimize ETL workflows for structured and unstructured data
- Work with Databricks Lakehouse architecture and implement best practices
- Manage and implement data governance using Unity Catalog
- Integrate AWS data services such as S3, IAM, VPC, and other relevant services
- Collaborate with cross-functional teams to deliver high-quality data solutions
- Build dashboards and support data visualization requirements
- Ensure data quality, reliability, and performance optimization
Requirements
- 4-8+ yrs experience
- Strong experience with Databricks platform
- Proficiency in Python for data engineering and ETL pipelines
- Hands-on experience with Unity Catalog
- Good knowledge of AWS services (S3, IAM, VPC, Glue/Lambda – preferred)
- Solid understanding of Data Lake / Lakehouse architecture
- Experience in building dashboards and reporting solutions
Tech Stack
- AWS
- ETL
- Python
- Unity