Senior Data Engineer, Auction Intelligence

Spotify is a leading audio streaming service on a mission to unlock the potential of human creativity. The Senior Data Engineer will be responsible for building scalable data pipelines and collaborating with machine learning engineers to support the advertising platform's auction intelligence systems.

Responsibilities:

• Design, build, and maintain scalable batch and streaming data pipelines using tools like Scio and GCP (BigQuery, Dataflow, GCS)
• Develop high-throughput, low-latency pipelines that support use cases such as budget pacing, campaign performance prediction, and auction bidding
• Partner closely with machine learning engineers and senior engineers to build data foundations for ML-driven products
• Improve data quality, reliability, and observability across pipelines to support critical business decisions
• Contribute to architecture and design decisions, balancing long-term scalability with practical delivery
• Write clean, maintainable, and well-tested code aligned with Spotify’s engineering practices
• Collaborate across functions with product managers, engineers, and stakeholders to deliver impactful solutions

Requirements:

• You have experience building and operating distributed data pipelines in production environments at scale
• You are experienced with data technologies such as BigQuery, Bigtable, or similar systems
• You write production-quality code in Java and/or Scala and have worked with frameworks like Scio or similar
• You have worked with orchestration tools such as Flyte, Styx, or comparable systems
• You are comfortable working with both batch and streaming data processing
• You approach problems with curiosity and use data to guide decisions and improvements
• You communicate clearly and collaborate effectively across technical and non-technical teams
• You are interested in working closely with machine learning systems or backend services
• Design, build, and maintain scalable batch and streaming data pipelines using tools like Scio and GCP (BigQuery, Dataflow, GCS)
• Develop high-throughput, low-latency pipelines that support use cases such as budget pacing, campaign performance prediction, and auction bidding
• Partner closely with machine learning engineers and senior engineers to build data foundations for ML-driven products
• Improve data quality, reliability, and observability across pipelines to support critical business decisions
• Contribute to architecture and design decisions, balancing long-term scalability with practical delivery
• Write clean, maintainable, and well-tested code aligned with Spotify's engineering practices
• Collaborate across functions with product managers, engineers, and stakeholders to deliver impactful solutions