AI Agent Developer & Tester

Role: AI Agent Developer & Tester

Location: 100% remote role

Duration: 8-month contract

Job Description:

Our client is seeking an AI/Agent Developer & Tester to support the development, testing, and deployment of AI agent workflows within their platform. This role will be responsible for building and validating LangGraph-based agents, developing automated test coverage, and ensuring reliability across development and production environments.

The ideal candidate will have strong experience with Python, LangGraph, and modern testing frameworks. Responsibilities include creating node-level unit tests, graph-level integration tests, regression testing, and production monitoring for AI-powered applications. This individual will also manage end-to-end UI testing using Playwright and support cloud-based infrastructure that powers the AI platform.

This role requires a quality-focused engineer who can work independently, build scalable testing practices, and help deliver stable, production-ready AI solutions.

Required Skills:

Experience with LangGraph and LangChain

Strong Python development experience

Experience with pytest and pytest-asyncio

Experience building unit tests for AI agents and workflow nodes

Experience creating end-to-end agent and graph integration tests

Knowledge of LLM mocking and test isolation strategies

Experience managing non-deterministic AI behavior through controlled testing methods

Experience designing and maintaining regression test suites

Playwright end-to-end testing experience

Test coverage reporting and analysis experience

Experience with JSON schema validation and structured outputs

Production monitoring and observability experience

Cloud application development and support experience (Google Cloud Platform preferred)

Strong troubleshooting and problem-solving skills

BonSoft Skills:

Experience with LangSmith or RAGAS

Prompt versioning experience

Adversarial or red-team testing experience

Token usage and cost monitoring experience

Experience with document extraction or OCR workflows

Experience in financial services or other regulated industries

Experience with Google Cloud Platform and Cloud Run

Terraform experience

Azure DevOps pipeline experience

Strong communication and collaboration skills