Senior Site Reliability Engineer

The Wikimedia Foundation is looking for a Senior Site Reliability Engineer to support and develop the platform serving Wikipedia to millions of people worldwide. The role involves ensuring the health of the infrastructure, automating processes, and collaborating with product teams to enhance service scalability.

Responsibilities:

• Performing day-to-day operational/DevOps tasks on Wikimedia’s public-facing infrastructure (deployment, maintenance, configuration, troubleshooting)
• Implementing and utilizing configuration management and deployment tools (Puppet, Kubernetes)
• Leading continuous improvement by automating the installation, configuration, and maintenance of services on our platform
• Work closely with product teams, helping them bring scalable functionality to our users by assisting in the architectural design of new services and making them operate at scale
• Participating in a 24/7 on-call rotation shared across the broader SRE team. This includes taking part in incident response, diagnosis, and follow-up on system outages or alerts across Wikimedia’s production infrastructure
• Collaborating with a global, cross-functional team in an asynchronous communication environment
• Mentoring peers in your areas of technical and operational strength
• Ability and willingness to travel 1-2 times a year for in-person events and team meetings

Requirements:

• 6+ years of experience in an SRE/Operations/DevOps role as part of a team
• Experience with shell and any scripting languages used in an SRE context (Python, Go, Bash, Ruby; we primarily use Python) and configuration management tools (Puppet, Ansible; we use Puppet)
• Experience with distributed caching systems, including their underlying algorithms and how to optimize their performance
• A thorough, protocol-level understanding of TCP/IP, HTTP, TLS, and DNS
• Experience with package management on Linux systems (we use Debian)
• Strong Linux system-level troubleshooting skills
• History of automating tasks and processes, identifying process gaps, and finding automation opportunities
• Strong English language skills (verbal and written) and ability to work independently, as an effective part of a globally distributed team working across multiple time zones
• Experience leading and participating in incident response and post-incident review rituals, with the goal of conducting root cause analysis and implementing preventive measures
• Ability and willingness to travel 1-2 times a year for in-person events and team meetings
• Experience with Linux kernel tuning for high-traffic loads
• Experience with high-performance HTTP(S) caching-proxy software, such as HAProxy, Varnish (Vinyl), Apache Traffic Server, Envoy, or Nginx
• Experience with the use, maintenance, and configuration of monitoring, metrics, and logging infrastructure (Prometheus, Grafana, etc.)
• Developing/contributing to Free and Open Source software, or being part of an open-source community (share your favourite pull requests!)
• Experience with LAMP stack technologies (PHP/HHVM, memcached/Redis) -- MediaWiki experience is a definite plus
• Experience with defining cross-team SLOs and their implementation