Senior Software Engineer (AI Inference & Runtime Platform)

AZX is hiring a Senior Software Engineer to build and operate its AI inference control plane and secure agent-sandboxing platform. The role covers model serving, Kubernetes infrastructure, isolated runtime systems, stateful data services, and the supporting control plane and tooling.

Responsibilities:

• Manage the serving tier for open-weight models: engine deployment and configuration, cold-start strategy, per-model SLOs, and upgrade/canary discipline
• Administer the Kubernetes layer for inference and sandbox workloads: operators and CRDs, autoscaling (KEDA/Karpenter-class), GPU scheduling and sharing, and node lifecycle
• Own the stateful data plane end to end with restore procedures that are regularly tested
• Oversee the sandbox runtime and its host-side control plane: lifecycle, exec, snapshot/fork, teardown, metering, and the threat model of the isolation boundary
• Direct the FastAPI control-plane services, Terraform/OpenTofu, Bicep, and the dashboards
• Run the layer the backend services team builds on, expect to debug into their services, and expect them to read your dashboards
• Manage the open-source posture: build to OSS standards and release as it matures, with reviewed PRs, real docs, and reproducible builds

Requirements:

• 5+ years of shipping production systems in a systems language. Rust is the house language, but polyglots are welcome — deep Go, C/C++, or Zig with genuine appetite for Rust counts. Async runtimes, memory-safety discipline, and debugging at the syscall boundary should be familiar territory
• Operated Kubernetes workloads that other people depended on — controllers or operators, scheduling, autoscaling, node lifecycle. You've been paged, and the experience changed how you build
• Strong ability to threat-model isolation boundaries (namespaces, cgroups, seccomp, hypervisors), including identifying what an untrusted guest could observe, forge, or exhaust, and applying security best practices for agentic execution — least privilege, no credentials in the sandbox, audit trails, and human approval on write actions
• Hands-on experience deploying or operating open-weight LLM serving infrastructure (vLLM/SGLang or similar), including packaging models into reliable, metered production endpoints
• Performance discipline in distributed systems: you measure before you optimize, and you can tell the story of a latency you killed with the numbers attached
• Practical depth in some of our core stack — Rust (tokio), Python/FastAPI, Kubernetes operators (controller-runtime/Kubebuilder/CRDs), KEDA, Karpenter, GPU device plugins/DRA — with a genuine willingness to research your way into the rest
• Familiarity with isolation technology (Firecracker, Kata, gVisor, or comparable), secrets management and egress control (Vault/KMS-class), and hosting stateful systems (vector stores like pgvector/Qdrant, graph stores like Neo4j) with backup and failover discipline
• Comfort operating across cloud and GPU substrates — AWS/Azure/GCP plus managed GPU clouds — using infrastructure-as-code (Terraform/OpenTofu, Bicep) and observability tooling (OpenTelemetry)
• Bachelor's Degree
• Master's is a plus
• Applicants must be currently authorized to work in the United States on a full-time basis
• We are unable to sponsor or take over sponsorship of employment visas at this time