Senior Grafana Engineer

The UVA VEC is seeking advanced Grafana practitioners to contribute to the development of high-quality evaluation workflows for AI systems. The role involves designing and executing multi-step workflows across dashboards, alerting systems, and data sources, with a focus on clarity, accuracy, and reproducibility.

Responsibilities:

• Design multi-step Grafana workflows including dashboards, alerts, and data source configurations
• Execute workflows in a live Grafana environment to generate reference implementations
• Develop structured task prompts with clearly defined and verifiable outcomes
• Implement programmatic validation scripts to assess task completion accuracy
• Analyze system-generated outputs and identify points of failure in execution
• Refine task difficulty and structure based on observed performance patterns
• Document workflows and evaluation criteria with precision and clarity

Requirements:

• Minimum 2 years of hands-on Grafana experience in production environments
• Strong proficiency with PromQL and observability data modeling
• Experience configuring dashboards, alerting pipelines, and data sources (e.g., Prometheus, InfluxDB)
• Ability to translate technical workflows into structured, testable instructions
• Proficiency in Python for writing validation or grading scripts
• Strong analytical skills for identifying execution gaps and failure modes
• Ability to work independently in a remote, asynchronous environment
• Availability for consistent weekly contribution (10–15 hours minimum)
• Experience with Grafana API automation
• Background in Kubernetes or infrastructure monitoring
• Familiarity with AI evaluation, benchmarking, or testing systems