LLM DevOps / AI Inference Engineer

LLM DevOps / AI Inference Engineer -AWS -remote

Remote

Short JD

• Build and manage AWS infrastructure using Terraform for production AI/LLM workloads.
• Deploy and manage self-hosted LLM inference containers on GPU infrastructure.
• Work with AWS EKS/ECS, EC2 GPU instances (G5/G6, P4d/P5), VPC, IAM, KMS, ECR, Secrets Manager, and CloudWatch.
• Optimize LLM inference using batching, quantization, KV cache, autoscaling, and multi-GPU strategies.
• Manage CUDA-based containers, GPU workloads, and Kubernetes node autoscaling.
• Build CI/CD pipelines and standardized processes for rapid model onboarding and deployment.
• Implement rate limiting, retries, backoff, logging, quota management, and cost attribution.
• Develop observability for latency, throughput, token usage, GPU utilization/cost, failures, and model performance.
• Support reliable and cost-efficient large-scale/burst AI workloads.
• Work with model providers including OpenAI, Anthropic, Gemini, and self-hosted models.
• Contribute to confidential computing / TEE architecture, including AWS Nitro Enclaves.
• Strong Python, AWS, Terraform, Kubernetes, and LLM infrastructure experience required.