Key skills
ApacheAWSAzureCloudETLGoogle Cloud PlatformPySparkPythonSparkSQLMachine LearningMLELTData EngineeringDatabricksApache SparkGCPGoogle CloudLeadershipStakeholder ManagementCommunication
About this role
Role Overview
- Design and implement end-to-end data and machine learning solutions using Databricks
- Develop and optimize data pipelines, ETL/ELT processes, and ML workflows
- Build, train, and deploy machine learning models for real-world business use cases
- Work with large-scale structured and unstructured datasets
- Collaborate with cross-functional teams to gather requirements and translate them into technical solutions
- Provide technical leadership and best practices in data engineering and ML architecture
- Optimize performance, scalability, and cost-efficiency of data platforms
- Ensure data quality, governance, and security standards are met
- Support troubleshooting, debugging, and continuous improvement of data and ML systems
- Stay updated with the latest advancements in Databricks and ML technologies
Requirements
- Strong hands-on experience with Databricks
- Proven experience in Machine Learning (model development, training, deployment)
- Proficiency in Python and SQL
- Experience with Apache Spark / PySpark
- Strong understanding of data engineering concepts and modern data architectures
- Experience working with cloud platforms (e.g., Azure, AWS, or GCP)
- Ability to design scalable and efficient data and ML solutions
- Strong problem-solving and analytical skills
- Excellent communication and stakeholder management skills.
Tech Stack
- Apache
- AWS
- Azure
- Cloud
- ETL
- Google Cloud Platform
- PySpark
- Python
- Spark
- SQL