Own the abuse vision for Arena: what gets detected, what gets enforced, how fast, and with what false-positive budget
Design and operate detection for bots, sybils, coordinated inauthentic voting, and rating-system manipulation — the integrity of Arena's leaderboards is the product
Build enforcement primitives (rate limits, challenges, shadowbans, account actions, model-side refusals) that are reversible, auditable, and humane
Detect and mitigate inference abuse and cost exploitation at the platform layer
Build jailbreak and multi-provider misuse detection across the models Arena serves, and partner with model-provider trust & safety teams on signal-sharing and escalation
Scope and implement abuse monitoring for every new product launch — web search, web fetch, live site deployment, and whatever's next — as part of the launch checklist, not after the fact
Prototype and mature into production systems of detection, review, and enforcement for the highest-severity harms (CSAM/NCII, violent extremism, self-harm), including the legal reporting pipeline (e.g., NCMEC)
Build internal investigator tooling so policy, on-call, and future T&S analysts can triage incidents without engineering bottleneck
Partner with Security on shared surface — account takeover, credential stuffing, API-key abuse, and the identity/behavioral-signal platform
Partner with policy, legal, and leadership on acceptable-use policy, enforcement escalations, and public-integrity narrative
Requirements
6+ years of production software engineering experience, including building and operating systems under adversarial conditions
Shipped experience in at least one of: trust & safety, anti-abuse, anti-fraud, anti-spam, integrity, or risk engineering
Strong SQL and data-analysis skills — this role is 30%+ pattern-finding and investigation, not just shipping code
Adversarial and investigative mindset — you can articulate a novel attack before designing the defense, and follow evidence when a novel harm surfaces
High judgment on false-positive cost, user harm, and the reversibility of enforcement actions
Proficiency in a modern backend language (Node.js, TypeScript, Python, or Go)
Excellent communication — you'll build alignment with engineering, product, policy, and leadership routinely.
Tech Stack
JavaScript
Node.js
Python
SQL
TypeScript
Go
Benefits
Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs.