Key skills
ETL designmaintenancePythonPySparkSQLDatabricksSnowflakeDBTGoAIETLdbtLeadershipMentoringCommunication
About this role
Afresh is the leading AI company in fresh food, partnering with grocers to order billions of dollars of fresh food nationwide. As a Senior Data Engineer, you'll play a key role in scaling and improving customer data integration and processing, designing ETLs, and contributing to new features that enhance the platform.
Responsibilities:
- Build tools and frameworks that streamline customer integrations, enabling faster onboarding and better handling of customer data
- Create robust ETLs in PySpark and DBT to process billions of records from customer datasets, ensuring data is accurate, reliable, and ready for downstream use
- Investigate and implement new technologies into the data platform, focusing on practical solutions that address current pain points and anticipate future needs
- Collaborate with product, engineering, and go-to-market teams to design and deliver data solutions for new products and features
- Identify and implement optimizations to improve ETL runtime and data processing scalability, reducing the time and effort required for integrations
- Solve real-world data quality challenges by working directly with messy, incomplete, or inconsistent customer data to extract the signal we need
- Support team members by mentoring engineers, leading technical discussions, and providing clear, actionable feedback
Requirements:
- Significant experience designing and maintaining ETLs that process large-scale datasets
- Proficiency with Python, PySpark, SQL, and experience working on platforms/tools like Databricks, Snowflake, or DBT
- Strong problem-solving skills and the ability to work with ambiguous or incomplete requirements to deliver concrete, impactful solutions
- A focus on practical outcomes—you're skilled at balancing technical rigor with the need to get things done
- Experience working directly with complex, unclean datasets and finding innovative ways to process and analyze them
- A knack for identifying areas where tooling or automation can simplify workflows and reduce manual effort
- Excellent communication skills—you're able to explain your ideas clearly to both technical and non-technical audiences
- Proven leadership in technical projects, with a willingness to mentor and help others grow