Agent product engineer

Judgment Labs builds learning infrastructure that helps AI agents improve through production experience, including traces, outcomes, failure modes, and evaluations. The Product Engineer will own end-to-end product development for agent investigations, verification workflows, interfaces, improvement loops, and supporting platform capabilities while working directly with customers.

Responsibilities:

• Judgment Agent: Shape how the Judgment Agent runs large-scale investigations: parallel investigators working across thousands of production traces, each covering a different dimension (failure modes, tool errors, regressions, drift), merging results into one answer
• Verification: Build the platform for verifying agent changes: hosted simulated environments for stateful agent evals, trajectory replay against changed agents, and monitors for unintended behavior changes
• Agent investigation interfaces: Design how engineers understand what their agents did and why. Long traces, tool calls, decisions, failures. What does debugging look like when the "program" is a reasoning loop? How do you make a thousand-step trajectory legible in minutes?
• Swarm UX: A hundred parallel investigations is useless if engineers can't follow them. Design how humans watch a swarm work, redirect investigators that go down the wrong path, and consume findings without reading a hundred reports
• The improvement loop: Build the workflows that turn production trajectories into datasets, judges, and regression checks, so the path from "found a problem" to "verified a fix" feels like one motion
• The platform underneath: Workspaces, roles, permissions, billing, usage, and limits for teams running many agents across many environments
• Judgment everywhere agents are built: An SDK and terminal-first experience so Claude Code, Codex, and OpenCode sessions can summon Judgment as a subagent mid-development