Key skills
AirflowAWSDockerETLKubernetesPySparkPythonJupyterData EngineeringUnit TestingS3GlueSageMakerGitVersion ControlPerformance OptimizationAgileScrumCommunicationCollaboration
About this role
Role Overview
- Assist in building and maintaining ETL/data pipelines using Python and PySpark
- Ingest, transform, and validate data from multiple sources
- Support data modeling and schema design for structured datasets
- Use Git for version control and collaborate with engineering teams
- Perform unit testing, code reviews, and performance optimization
- Contribute to technical documentation of data workflows and pipelines
- Support feature testing and controlled releases in QA/dev environments
- Perform exploratory analysis using Jupyter/Amazon SageMaker notebooks
- Work in a Scrum/Agile environment with clear communication and collaboration
Requirements
- Bachelor’s degree in Computer Science, Data Engineering, Data Science, or Statistics
- Experience in Python and PySpark (0–2 years)
- Basic knowledge of Airflow, AWS S3, and AWS Glue
- Familiarity with Git and Jupyter notebooks
- Understanding of Docker/Kubernetes concepts
Tech Stack
- Airflow
- AWS
- Docker
- ETL
- Kubernetes
- PySpark
- Python
Benefits
- Strong attention to detail, willingness to learn, and ability to work in a collaborative team environment
- Diversity, equity, and inclusion initiatives
- WE@IN women empowerment program
- Professional development opportunities