Software Engineer – Cloud Infrastructure

FriendliAI is an inference cloud company building infrastructure for running frontier open-weight AI models in production at scale. The Software Engineer, Cloud Infrastructure will architect and operate multi-tenant Kubernetes clusters, GPU scheduling and autoscaling systems, and the networking and reliability foundations supporting latency-sensitive inference workloads.

Responsibilities:

• Own the architecture of our multi-cluster, multi-tenant Kubernetes fleet across both managed and self-managed clusters: cluster topology, control plane and etcd lifecycle, and zero-downtime upgrades
• Extend Kubernetes with custom controllers, operators, and CRDs so platform behavior is encoded in software rather than runbooks
• Design GPU scheduling and capacity strategy, including topology-aware placement, node pools, priority and preemption, and quota across tenants
• Build autoscaling that matches inference traffic: queue-driven pod scaling, node autoscaling, scale-to-zero, and cold-start reduction
• Own the Kubernetes network data plane: CNI, IPAM, DNS, ingress, and L4/L7 load balancing
• Design cross-AZ, cross-region, and cross-cluster connectivity, and operate the service mesh for routing, mTLS, and traffic policy
• Debug production network issues (packet loss, conntrack exhaustion, MTU mismatches, DNS latency, load balancer behavior) and drive permanent fixes
• Define SLOs for platform-critical systems and lead post-incident hardening
• Deliver infrastructure as code with Terraform, Helm, and GitOps
• Partner with the inference engine, platform, SRE, and security teams to turn serving requirements into platform capabilities