LLM DevOps / Inference Engineer

Xaxis Solutions is seeking an LLM DevOps / Inference Engineer to support a healthcare-focused AI benchmark and evaluation platform. The role owns infrastructure for the benchmark harness, self-hosted model serving stack, and platform reliability, including provisioning GPU infrastructure, optimizing model inference, and managing AWS cloud systems.

Responsibilities:

• Build and maintain AWS infrastructure using Terraform, including networking, compute, orchestration, secrets, observability, and CI/CD
• Deploy and manage self-hosted LLM inference workloads on GPU infrastructure
• Implement batching, quantization, autoscaling, and multi-GPU strategies for efficient inference
• Develop standardized processes for onboarding new models quickly
• Build provider abstraction layers for API-based models such as OpenAI, Anthropic, and Gemini, as well as self-hosted models
• Implement rate limiting, retries, backoff, quota management, request/response logging, and cost attribution
• Ensure benchmark runs are reproducible and cost-efficient through pinned model/container versions and optimized cloud capacity
• Build observability dashboards covering throughput, latency, token usage, GPU-hour costs, failures, and per-model performance
• Support burst workloads while preventing orphaned or unnecessary cloud resources
• Contribute to Trusted Execution Environment (TEE) architecture and evaluate AWS Nitro Enclaves and comparable confidential-computing solutions

Requirements:

• Hands-on experience launching and managing inference containers
• Strong experience building and maintaining AWS infrastructure as code, including networking, compute, orchestration, secrets, observability, and CI/CD
• Experience working in a startup or fast-paced environment
• Strong AI/LLM and model infrastructure experience
• Excellent communication skills
• US Citizens will be given first preference
• Active and complete LinkedIn profile
• Production-scale AWS infrastructure experience
• Strong knowledge of EKS/ECS, EC2 GPU instances (G5/G6, P4d/P5), VPC, IAM, KMS, Secrets Manager, ECR, CloudWatch, and Service Quotas
• Strong Terraform / Infrastructure as Code experience
• Hands-on experience with LLM model serving and inference optimization
• Practical knowledge of batching strategies, KV cache behavior, quantization, and multi-GPU sharding
• Experience managing GPU workloads with EKS/ECS, node autoscaling, and CUDA-based container pipelines
• Strong reliability and cost-engineering experience, including SLOs, monitoring, alerting, and cloud cost optimization
• Experience with AWS SageMaker and Bedrock
• Strong Python development skills
• Knowledge of confidential computing, including Enclaves, remote attestation, and sealed key release
• Understanding of healthcare compliance, including HIPAA-eligible services, BAA requirements, audit logging, and PHI access controls
• Experience with security hardening, image scanning, and network egress controls