Senior Site Reliability Engineer

Role Overview

• Lead and refine the incident lifecycle: detection, triage, communication, mitigation, resolution, and post-incident review.
• Define and maintain severity models, escalation paths, on-call expectations, and runbooks/playbooks—keeping them current and usable under pressure.
• Facilitate blameless postmortems; turn findings into tracked remediations and shared learning that reduces repeat incidents.
• Improve coordination during major incidents: roles, tooling, customer/stakeholder updates, and handoffs.
• Partner with security, support, and product on incident communications and regulatory or contractual obligations where applicable.
• Establish and maintain organization-wide standards for metrics, logs, and traces in Datadog—including naming conventions, cardinality, retention, and sampling—so teams can instrument consistently and confidently.
• Define and drive adoption of SLOs, SLIs, and error budgets across engineering teams; meet teams where they are—bootstrapping SLI/SLO programs for teams starting from scratch and improving rigor for teams that already have them, with the long-term goal of teams owning their own observability.
• Build and maintain reusable Datadog dashboard templates, monitor templates, and alerting patterns that teams can adopt and adapt—reducing the activation energy for doing observability well.
• Champion golden signals and RED/USE-style alerting philosophies; align alerts with user-impacting symptoms, not just low-level infrastructure noise.
• Partner with the Infrastructure team on observability stack decisions, multi-tenancy, cost controls, and data lifecycle.
• Continuously reduce alert noise through threshold tuning, ownership assignment, and on-call load management.
• Mentor engineers on operational excellence, safe deployment practices, and production readiness; help engineering teams grow their own reliability instincts.
• Contribute to capacity planning, chaos/game-day exercises, and reliability reviews for critical changes.
• Serve as a connective layer between the SRE and Infrastructure teams—aligning on tooling, standards, and shared goals.

Requirements

• Experience: 5+ years in SRE, production engineering, or equivalent, including on-call responsibility for customer-facing systems.
• Incidents: Proven experience running or significantly improving incident response (process, tooling, or both) in a distributed systems environment.
• Observability: Deep, hands-on experience with Datadog—building dashboards, monitors, and instrumentation standards across multiple teams or services. Experience with metrics, logging, and tracing at scale.
• SLI/SLO Programs: Demonstrated experience defining SLOs/SLIs and error budget policies in production; comfortable working with teams to codify the metrics their reliability posture is based on.
• Systems: Strong understanding of Linux, networking, distributed systems failure modes, and cloud or hybrid infrastructure (Kubernetes, load balancers, databases, queues).
• Automation: Proficiency in at least one of Go, Python, or similar for tooling and automation; comfort with IaC concepts (Terraform or equivalent).
• Communication: Clear written and verbal communication; ability to facilitate discussions during high-pressure incidents and deliberate postmortems alike.
• Collaboration: Track record of influencing without direct authority and driving adoption across engineering teams.

Tech Stack

• Cloud
• Distributed Systems
• Kubernetes
• Linux
• Python
• Terraform
• Go

Benefits

• Competitive salary
• Hybrid work environment (3 days in office per week)
• 100% individual and dependent medical + dental + vision coverage
• 401(K) with a 4% company match
• 20 days PTO
• Iru Wellness Week the first week in July
• Equity for full-time employees
• In-office lunch stipend provided
• Up to 16 weeks of paid leave for new parents
• Paid Family and Medical Leave
• Modern Health mental health benefits for individuals and dependents
• Fertility benefits
• Working Advantage employee discounts
• Onsite fitness center
• Free parking
• Exciting opportunities for career growth