Senior Inference Optimization Engineer - Dragonfly Portfolio

Dragonfly is a crypto-native venture capital and research firm that connects candidates with roles across its portfolio companies. It is sourcing a Senior Inference Optimization Engineer to optimize large language model inference infrastructure, improving throughput, latency, and cost efficiency at scale. The role involves GPU infrastructure, benchmarking, distributed inference, load balancing, and evaluation of emerging optimization techniques and hardware.

Responsibilities:

• Stand up and optimize GPU infrastructure including B300 nodes in owned data centers
• Drive down TTFT and TPOT, push throughput, and improve cost per token for LLM inference workloads
• Build reproducible benchmarking harnesses across inference engines to identify optimal engine, quantization scheme, and parallelism strategy per workload and GPU SKU
• Optimize multivariate inference load-balancing algorithms within the inference routing system
• Evaluate emerging inference optimization techniques including custom CUDA/Triton kernels, novel attention variants, new quantization schemes, and compilation stack improvements
• Evaluate emerging inference hardware (FPGAs, ASICs, custom silicon) for viability in the stack