Senior Data Engineer

Thrive Market is an online, membership-based market focused on making healthy and sustainable living accessible and affordable. The Senior Data Engineer will build and deliver data engineering solutions, managing the full lifecycle from ingestion and processing through ETL and reporting delivery while partnering with cross-functional technical and business teams. The role also focuses on data architecture, infrastructure reliability, data integration, and solving production data issues.

Responsibilities:

• Work across multiple projects and efforts to orchestrate and deliver cohesive data engineering solutions in partnership with various functional teams at Thrive Market
• Be hands-on and take ownership of the complete cycle of data services, from data ingestion, data processing, and ETL to data delivery for reporting
• Collaborate with other technical teams to deliver data solutions that meet business and technical requirements; define technical requirements and implementation details for the underlying data lake, data warehouse, and data marts
• Identify, troubleshoot, and resolve production data integrity and performance issues
• Collaborate with all areas of data management as lead to ensure patterns, decisions, and tooling are implemented in accordance with enterprise standards
• Perform data source gap analysis and create data source/target catalogs and mappings
• Develop a thorough knowledge and understanding of cross-system integration, interactions, and relationships in order to develop an enterprise view of Thrive Market’s future data needs
• Design, coordinate, and execute pilots/prototypes/POC to provide validation on specific scenarios and provide an implementation roadmap
• Recommend/Ensure technical functionality (e.g. scalability, security, performance, data recovery, reliability, etc.) for Data Engineering
• Facilitate workshops to define requirements and develop data solution designs
• Apply enterprise and solution architecture decisions to data architecture frameworks and data models
• Maintain a repository of all data architecture artifacts and procedures
• Collaborate with IT teams, software providers, and business owners to predict and devise data architecture that addresses business needs for collection, aggregation, and interaction with multiple data streams

Requirements:

• Excellent communication and presentation skills (verbal, written, presentation) across all levels of the organization. Ability to translate ambiguous concepts into tangible ideas
• Hands on experience  programming in Python, DBT and Airflow
• Expertise with RDBMS and Data Warehousing (Strong SQL) with Snowflake or similar
• In-depth knowledge and experience with data and information architecture patterns and implementation approaches for Operational Data Stores, Data Warehouses, Data Marts and Data Lakes
• Experience administering and managing data infrastructure using Terraform
• Proficiency in logical/physical data architecture, design and development
• Experience in Data lake / Big data analytics platform implementation either cloud based or on-premise; AWS preferred
• Experience working with high volumes of data; experience in design, implementation and support of highly distributed data applications
• Experience with Development Tools for CI/CD, Unit and Integration testing, Automation and Orchestration E.g. GitHub, Jenkins
• Experience with writing Kafka producers and consumers or experience with AWS Kinesis
• A knack for independence (hands-on) as well as team work
• Excellent analytical and problem-solving skills, often in light of ill-defined issues or conflicting information
• Adept in the  ability to elicit, gather, and manage requirements in an Agile delivery environment
• This position requires traveling to our HQ office in Los Angeles, California, twice a year for all-company summits; once in the summer and once in the winter
• Employment with Thrive Market requires that employees be based in the United States. This is a condition of employment and must be maintained throughout the duration of employment
• Experience evaluating and working with tools and vendors that support marketing measurement frameworks, including MMM and MTA, is a plus
• Experience using Claude or similar AI agents for development is a plus
• Experience with streaming data ingestion, machine-learning, Apache Spark a plus