AI Inference Engineer

F5 is a company focused on helping organizations create, secure, and run applications while advancing cybersecurity and digital experiences. The AI Inference Engineer will optimize Large Language Models for high-performance inference across data centers and edge devices, maximizing throughput, minimizing latency, and maintaining accuracy. The role also involves designing scalable infrastructure, leveraging hardware acceleration, and monitoring enterprise-grade AI system performance.

Responsibilities:

• Build and maintain robust inference engines using tools like vLLM, TGI (Text Generation Inference), and NVIDIA Triton, ensuring high performance at scale
• Handle deployment optimizations to deliver low-latency AI serving solutions for multiple business applications
• Profile and optimize models for specialized hardware backends, including NVIDIA GPUs (CUDA/TensorRT), Apple Silicon (CoreML), and AI accelerators like TPUs and LPUs
• Collaborate with hardware teams to maximize utilization and performance across various computational environments
• Design and implement auto-scaling architectures for online (real-time) and batch inference pipelines, leveraging Kubernetes for inference routing and orchestration
• Ensure software solutions are optimized for peak performance during traffic spikes, maintaining reliability and scalability
• Establish robust observability frameworks to monitor Time to First Token (TTFT), tokens per second, and memory bandwidth utilization against service-level agreements (SLAs)
• Build and execute performance and load testing suites to identify bottlenecks and ensure consistent reliability at scale