Senior Software Engineer, ML API – AI Inference

Cerebras is hiring a Senior Software Engineer to build and evolve the ML API layer for its disaggregated AI inference systems. The role owns inference API, model integration, and request-routing features, delivering reliable, consistent serving across models and accelerator backends.

Responsibilities:

• Own features end to end. Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities
• Lead multi-component delivery. Write clear design documents, evaluate technical tradeoffs, break projects into deliverable milestones, and coordinate implementation with a small group of engineers and partner teams
• Deliver consistent API behavior. Implement clear request and response semantics across GPU prefill, Cerebras decode, and other inference backends. Maintain compatibility with widely adopted inference interfaces and evolve owned APIs through validation, versioning, and deprecation best practices
• Enable new models and capabilities. Integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific features into the serving platform
• Integrate serving components. Extend inference services and their integration with frameworks and runtimes such as vLLM, PyTorch, Hugging Face libraries, LLGuidance, and Cerebras runtime components
• Support disaggregated inference. Implement routing, state-transfer integration, error handling, retries, cancellation, and request lifecycle management to coordinate GPU prefill with Cerebras decode
• Deliver measurable performance improvements. Diagnose bottlenecks and improve time to first token, streaming latency, throughput, and resource efficiency through changes to tokenization, serialization, batching, scheduling, and communication
• Own correctness and release quality. Build conformance tests, model-validation suites, numerical comparisons, integration tests, and performance benchmarks that catch regressions in owned features and support safe releases
• Strengthen production reliability. Improve service metrics, tracing, logging, alerts, and diagnostic tools. Investigate production incidents, resolve root causes, and follow through on improvements to resilience and incident response
• Improve customer and developer experience. Partner with product, support, and solutions teams to understand user needs and make practical tradeoffs. Improve configuration, documentation, examples, and debugging workflows
• Raise the engineering quality bar. Provide thoughtful design and code reviews, mentor less-experienced engineers, share domain knowledge, and proactively remove delivery roadblocks