Build and maintain scalable data pipelines to process and integrate e-commerce data from multiple sources
Develop production-ready Python scripts for API consumption, web scraping, and data transformation
Design and implement cloud-based data architecture using AWS services such as S3, EC2, and Lambda
Orchestrate and monitor scheduled data collection workflows using Apache Airflow
Evaluate and implement tools for high-performance data transformation and analysis (e.g., Polars, DuckDB, PySpark)
Collaborate with data analysts to deliver accessible, well-structured datasets for reporting and advanced analytics
Identify opportunities to improve data quality, reliability, and performance across all data collection endpoints

Proficiency in programming languages such as Java and Python
Hands-on experience with SQL and database design
Previous experience as a Data Engineer or in a similar role
Strong technical knowledge of web data collection architectures, including proxy and header management
Excellent numerical and analytical skills
Willingness to learn and adapt to new tools and technologies
Proactive, curious, and highly tenacious, with a strong drive to grow and stay ahead of industry trends
Comfortable navigating a fast-paced, ambiguous environment with a high degree of independence
A Bachelor’s degree in a quantitative field (e.g., Computer Science, Engineering, Information Systems) is preferred

Data Engineer, Data Collection Team

Key skills