Senior Site Reliability Engineer

Zocdoc is a digital health marketplace that helps patients find and book care. The Senior Site Reliability Engineer operates and improves production infrastructure, observability, incident response, and recovery, while automating operational work and partnering with engineering teams to keep production reliable and safe.

Responsibilities:

• Operating and evolving our observability stack (Datadog, Sentry, Honeycomb, OpenTelemetry pipelines), including alerts, SLIs, and SLOs that page the right owner
• Serving in the on-call and incident-command rotation, and driving follow-up work from retrospectives to completion
• Provisioning and maintaining AWS accounts and ECS/EKS clusters with our cloud engineering partners, and right-sizing capacity so we meet reliability targets without over-provisioning
• Running and hardening the network edge: CDN, bot protection, API gateway, and WAF
• Designing and testing failover and recovery for compute, network, and edge, including game days and DR exercises
• Applying infrastructure as code to production with guardrails, drift detection, and safe rollback paths
• Applying and improving our rollout policy and safety gates, and partnering with product teams on readiness and failure modes
• Keeping production infrastructure patched and certificates current, and automating that work so it doesn't come back every quarter
• Maintaining security posture for production infrastructure (least-privilege IAM, org-level guardrails, edge protection, container image and dependency scanning) in partnership with Security
• Mentoring engineers through incident reviews, code review, and pairing, and writing runbooks so routine requests don't need you

Requirements:

• 5+ years in Site Reliability, Production, Platform, or DevOps Engineering supporting consumer-facing web applications or distributed production systems
• 2+ years of on-call in a 24/7 cloud production environment, including leading incident response and delivering durable remediation
• Hands-on experience building observability, including structured logging, distributed tracing, alerting, and SLOs
• Experience designing for failure in distributed systems, such as timeouts, retries, load shedding, and graceful degradation
• Production experience with AWS and container platforms such as ECS, EKS, or Kubernetes
• Solid Linux fundamentals, including patching, performance troubleshooting, and container internals
• Strong networking and performance fundamentals: TCP/IP, HTTP, TLS, DNS, load balancing, and VPC design, and tracking down latency and resource bottlenecks
• Deep hands-on experience in at least one of: the network edge (CDN, API gateway, WAF), infrastructure as code and CI/CD (Terraform, CDK, or CloudFormation), or automating operational toil such as patching, certificates, or image pipelines
• Proficiency in Python, Go, or a comparable language, and the ability to read, review, and debug application code
• Sound judgment using AI tools in engineering and operations, including validating AI-generated changes before they reach production
• Clear written and verbal communication, including explaining operational risk to engineers and non-technical partners
• Depth in more than one of the areas above
• Experience with error budgets, failover design, DR exercises, or game days
• Experience with a CI/CD system such as GitHub Actions, Harness.io, CircleCI, or TeamCity
• Hands-on experience with tools we run, such as Datadog, Sentry, Honeycomb, Rootly, Fastly, or Kong
• Windows Server experience, since our production includes a Windows fleet alongside containerized AWS services
• Experience in regulated environments such as healthcare