ML Runtime Engineer

Fractile develops specialized hardware and software systems to accelerate AI inference. The ML Runtime Engineer will integrate Fractile's AI acceleration hardware with leading inference engines, develop KV cache management solutions, and help build a scalable reference inference engine focused on transformer models.

Responsibilities:

• As an ML Runtime Engineer you will integrate Fractile's AI acceleration hardware with leading inference engines including vLLM and SGLang, research and build proof-of-concept KV cache management implementations tailored to our hardware, and work closely with the broader runtime team to design and build a scalable reference inference engine
• You will focus primarily on the transformer ML architecture and share your expertise to help shape the direction of our runtime stack