Senior Data Engineer

Thrive Market is an online, membership-based market dedicated to making healthy and sustainable living affordable for everyone. They are seeking a Senior Data Engineer to build high-impact products, focusing on data processing and analytics initiatives that provide value to their members.

Responsibilities:

• Work across multiple projects and efforts to orchestrate and deliver cohesive data engineering solutions in partnership with various functional teams at Thrive Market
• Be hands-on and take ownership of the complete cycle of data services, from data ingestion, data processing, and ETL to data delivery for reporting
• Collaborate with other technical teams to deliver data solutions that meet business and technical requirements; define technical requirements and implementation details for the underlying data lake, data warehouse, and data marts
• Identify, troubleshoot, and resolve production data integrity and performance issues
• Collaborate with all areas of data management as lead to ensure patterns, decisions, and tooling are implemented in accordance with enterprise standards
• Perform data source gap analysis and create data source/target catalogs and mappings
• Develop a thorough knowledge and understanding of cross-system integration, interactions, and relationships in order to develop an enterprise view of Thrive Market’s future data needs
• Design, coordinate, and execute pilots/prototypes/POC to provide validation on specific scenarios and provide an implementation roadmap
• Recommend/Ensure technical functionality (e.g. scalability, security, performance, data recovery, reliability, etc.) for Data Engineering
• Facilitate workshops to define requirements and develop data solution designs
• Apply enterprise and solution architecture decisions to data architecture frameworks and data models
• Maintain a repository of all data architecture artifacts and procedures
• Collaborate with IT teams, software providers, and business owners to predict and devise data architecture that addresses business needs for collection, aggregation, and interaction with multiple data streams

Requirements:

• Excellent communication and presentation skills (verbal, written, presentation) across all levels of the organization. Ability to translate ambiguous concepts into tangible ideas
• Hands on experience programming in Python, DBT and Airflow
• Expertise with RDBMS and Data Warehousing (Strong SQL) with Snowflake or similar
• In-depth knowledge and experience with data and information architecture patterns and implementation approaches for Operational Data Stores, Data Warehouses, Data Marts and Data Lakes
• Experience administering and managing data infrastructure using Terraform
• Proficiency in logical/physical data architecture, design and development
• Experience in Data lake / Big data analytics platform implementation either cloud based or on-premise; AWS preferred
• Experience working with high volumes of data; experience in design, implementation and support of highly distributed data applications
• Experience evaluating and working with tools and vendors that support marketing measurement frameworks, including MMM and MTA, is a plus
• Experience with Development Tools for CI/CD, Unit and Integration testing, Automation and Orchestration E.g. GitHub, Jenkins
• Experience using Claude or similar AI agents for development is a plus
• Experience with writing Kafka producers and consumers or experience with AWS Kinesis
• A knack for independence (hands-on) as well as team work
• Excellent analytical and problem-solving skills, often in light of ill-defined issues or conflicting information
• Experience with streaming data ingestion, machine-learning, Apache Spark a plus
• Adept in the ability to elicit, gather, and manage requirements in an Agile delivery environment