Staff Software Engineer, ML Infra & Distributed Systems

Tubi is a free streaming service that provides personalized entertainment and content discovery experiences. The Staff Software Engineer will build and operate scalable, low-latency machine learning inference platforms and distributed systems supporting recommendations, search, advertising, and content understanding. The role also involves architectural leadership, cross-functional collaboration, incident management, optimization, and mentoring.

Responsibilities:

• Design and build scalable, high throughput, and low latency distributed systems using Scala
• Build reusable components and services that serve various ML applications like Personalization, Search, Ads and Exploration
• Partner closely with ML engineers to understand their challenges and limitations and develop scalable solutions to address them. Proactively recommend solutions to keep our ML Inference stack state of the art
• Take a data driven approach to identifying & optimizing latency, cost, and efficiency of our infra. Lead large scale cross functional refactorings if necessary
• Mentor other engineers on the team on system design, effective incident management, interviewing, leveraging LLMs for work, etc
• Collaborate with ML, Product, and cross functional engineering teams to define the long term vision and architecture for ML Infrastructure at Tubi

Requirements:

• Experience designing and building scalable, distributed systems in any modern backend language (e.g., Scala, Java, Python, Go, C++); experience with Scala or JVM based language is a plus
• Strong experience with AWS or an equivalent cloud platform
• Experience building online microservices at scale with low latency serving
• Experience with both SQL (e.g. Postgres) and NoSQL databases (e.g. Cassandra), message brokers (e.g. Kafka), and caches (e.g. Redis)
• Experience with containerization technologies, such as Docker or Kubernetes
• Led the response and resolution efforts for multiple major, large-scale incidents
• Familiarity with the machine learning infrastructure like inference engines (e.g. torschserve, triton, vLLM), vector stores (e.g. LanceDB, FAISS), feature stores (e.g. Feast), ElastiCache, model training orchestration, etc
• Understanding of ML model training pipelines and model internals. Experience with Recommender Systems, Search, Autocomplete and Ads ML is a plus
• Previous experience with Akka, Erlang, Elixir or Go
• Proficient in data-driven analysis of complex A/B testing results