LLM Inference Engineer

Majestic Labs is hiring an LLM Inference Engineer to own the end-to-end LLM serving stack on its custom hardware. The role focuses on adapting and optimizing inference frameworks, building runtime and distributed inference capabilities, and identifying performance bottlenecks to inform kernel, compiler, and hardware development.

Responsibilities:

• The serving stack, end to end — bring up and adapt a modern inference framework (vLLM, SGLang, or similar) to run on Majestic hardware
• The runtime hot path — continuous batching, the scheduler, paged KV cache, and prefill/decode disaggregation
• Distributed inference at scale — tensor, pipeline, and expert parallelism across accelerators, wired into our collective communication library (CCL)
• The multi-modal pipeline — image, audio, and video preprocessing, encoder integration, and mixed-modality batching
• Inference-time techniques — speculative decoding, prefix caching, and structured decoding
• End-to-end performance — profile, benchmark, and hunt down bottlenecks across the full serving path, feeding findings back to the kernel, compiler, and hardware teams