Senior Data Engineer

Brooksource is a pioneering marketing technology company seeking a collaborative and innovative Senior Data Engineer to manage data pipelines and real-time streaming for actionable insights. The role involves optimizing data processes, managing the Snowflake data warehouse, and supporting new machine learning initiatives.

Responsibilities:

• Manage, maintain, and scale numerous data pipelines transmitting real-time data used to find insights and take action with customers
• Manage the Snowflake data warehouse, optimizing queries and storage performance
• Intensive SQL-querying to optimize pipelines and databases, manage and validate data, and more
• Identify and solve issues in storage, streaming, and pipelines; guide other engineers in best practices, and support new ML initiatives in predictive analytics and beyond
• Explore and learn more about the industry and space, contributing to architectural design and exploring predictive feature-building and data-driven optimizations

Requirements:

• Significant expertise in complex SQL concepts, transforming data using dbt, and optimizing storage and pipelines
• Expertise in using Python to write scripts for data manipulation, analysis, validation, and automation
• Expertise in Snowflake data warehouse usage and management, including writing stored procedures, maintaining streaming pipelines, and de-duplication
• Hands-on responsibilities supporting real-time data streaming pipelines using AWS Kinesis or Kafka
• Significant exposure to enterprise AWS data environments using Kinesis, Lambda, and cloud databases
• Experience using and managing CI/CD pipelines and version control tools to manage and deploy changes