Databricks - Data Engineer

Applexus Technologies is hiring a Databricks Data Engineer to build and optimize data pipelines and implement data engineering solutions using AWS and Databricks. The role also focuses on automating data workflows and developing SQL analytics while supporting data integrity and performance.

Responsibilities:

• Data Pipeline Development: Design, build, and optimize scalable data pipelines. AWS Cloud Expertise: familiar with analytical services S3,Glue,redshift, Lambda
• Databricks Implementation: Leverage Databricks for data engineering tasks, including ETL/ELT processes, data lake management, advanced analytics, Unity catalog and CICD Deployment
• Coding and Automation: Write clean, efficient, and maintainable code in Python, Scala, or Java to automate data workflows and integrate with various systems
• SQL Analytics: Develop complex SQL queries for data analysis, reporting, and optimization, ensuring data integrity and performance. Experience with Data Modelling & Performance Optimization techniques

Requirements:

• Candidate should be proficient in AWS and Databricks, possessing strong coding and SQL analytical skills, and adept at handling large volumes of unstructured data
• familiar with analytical services S3,Glue,redshift, Lambda
• Write clean, efficient, and maintainable code in Python, Scala, or Java
• Develop complex SQL queries for data analysis, reporting, and optimization, ensuring data integrity and performance
• Experience with Data Modelling & Performance Optimization techniques
• Should possess minimum of 8+ years of experience in data engineering, with focus on Databricks/Spark Coding based implementations
• Proficient in programming languages such as Python, pyspark for data pipeline development
• Advanced SQL skills for complex querying, optimization, and data transformation
• Familiarity with CI/CD pipelines, version control (e.g., Git), and infrastructure-as-code (e.g., Terraform, Azure Devops)
• Strong problem-solving and analytical skills
• Ability to work in a fast-paced, agile environment and manage multiple priorities
• Experience with containerization (e.g., Docker, Kubernetes) for deploying data applications would be an added advantage