Senior Data Engineer

Afresh is an AI platform for grocery that focuses on reducing food waste and improving decision-making for grocers. As a Senior Data Engineer, you will design and implement ETLs to process customer data and improve integration processes, contributing to the development of new features in the platform.

Responsibilities:

• Build tools and frameworks that streamline customer integrations, enabling faster onboarding and better handling of customer data
• Create robust ETLs in PySpark and DBT to process billions of records from customer datasets, ensuring data is accurate, reliable, and ready for downstream use
• Investigate and implement new technologies into the data platform, focusing on practical solutions that address current pain points and anticipate future needs
• Collaborate with product, engineering, and go-to-market teams to design and deliver data solutions for new products and features
• Identify and implement optimizations to improve ETL runtime and data processing scalability, reducing the time and effort required for integrations
• Solve real-world data quality challenges by working directly with messy, incomplete, or inconsistent customer data to extract the signal we need
• Support team members by mentoring engineers, leading technical discussions, and providing clear, actionable feedback

Requirements:

• Significant experience designing and maintaining ETLs that process large-scale datasets
• Proficiency with Python, PySpark, SQL, and experience working on platforms/tools like Databricks, Snowflake, or DBT
• Strong problem-solving skills and the ability to work with ambiguous or incomplete requirements to deliver concrete, impactful solutions
• A focus on practical outcomes—you're skilled at balancing technical rigor with the need to get things done
• Experience working directly with complex, unclean datasets and finding innovative ways to process and analyze them
• A knack for identifying areas where tooling or automation can simplify workflows and reduce manual effort
• Excellent communication skills—you're able to explain your ideas clearly to both technical and non-technical audiences
• Proven leadership in technical projects, with a willingness to mentor and help others grow