Design, implement, and maintain scalable data pipelines and ETL/ELT workflows on Databricks and cloud platforms.
Manage large-scale geospatial and temporal datasets stored in AWS S3.
Collaborate with data scientists to productionize machine learning models and ensure smooth data availability.
Implement data validation, testing, and monitoring frameworks to ensure data accuracy, consistency, and reliability.
Optimize data storage and processing strategies to handle high volumes of traffic and mobility data efficiently.
Develop and maintain documentation for data workflows, architecture, and processes.
Work closely with cross-functional teams to understand data requirements and ensure timely delivery.
Stay up-to-date with the latest trends and best practices in data engineering, cloud technologies, and big data processing.

Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
3+ years of experience as a data engineer or in a similar role.
Strong proficiency in Python and associated libraries for data engineering (pandas, PySpark, etc.).
Hands-on experience with Databricks and Spark for large-scale data processing.
Experience with AWS services, especially S3, and knowledge of cloud-based data architectures.
Solid understanding of data pipeline testing, version control, and CI/CD practices.
Experience with SQL and NoSQL databases.
Strong problem-solving skills and attention to detail.

Data Engineer

Key skills