Rubric Quality Researcher — Research Replication

Infinity Artificial Intelligence Institute develops research benchmarks for measuring progress toward autonomous AI research. The Rubric Quality Researcher will improve automated research-task graph generation, design efficient evaluation methods, defend claim-anchored quality metrics, audit generated graphs, and curate the research corpus for publication.

Responsibilities:

• Iterate on the generation pipeline (outline-then-subtree chunked generation, critique/fix convergence loops, deterministic quality gates) until generated graphs are a credible substitute for expert ones
• Design and run run-free evaluation ladders so iteration doesn't require $500 agent runs: claim-coverage joins, LLM-judge alignment (recall/precision vs. expert rubrics), retrospective grading against frozen agent implementations
• Own the claim-anchored success metric: the number of a paper's claims an agent independently re-measures under a generated graph must meet or beat the expert-rubric baseline. Defend the metric against inflation (rubric authors control leaf counts; they must not control the denominator)
• Ground graphs in real artifacts (reference repos, datasets, addenda) and build the audits that catch padding, duplication, and hallucinated tasks
• Expand and curate the corpus; help publish the methodology