Software Developer

Role Overview

• Design and ship multi-step AI agents using modern orchestration frameworks (Claude, OpenAI Agents SDK, or equivalent), including prompt design, state management, tool calling, and human-in-the-loop control.
• Build and maintain MCP servers and tool integrations connecting agents to internal services, data warehouses, and third-party APIs; define clean schemas, error handling, and least-privilege authorization scopes.
• Implement retrieval-augmented generation (RAG) pipelines — ingestion, chunking, embedding, hybrid retrieval, reranking — grounded in Guideline’s proprietary spend, pricing, and media datasets.
• Develop offline and online evaluations (LLM-as-judge, deterministic checks, golden sets, regression suites) that measure agent quality, tool-use correctness, task completion, latency, and cost before each release.
• Instrument agents with end-to-end tracing and observability (e.g., OpenTelemetry, LangSmith, MLflow) and operate them in production: monitor drift, regressions, prompt-injection attempts, and hallucination rates.
• Apply security and safety controls — input/output filtering, prompt-injection defenses, sandboxed tool execution, PII handling, data residency — in collaboration with Security and Compliance.
• Optimize for cost and latency through model routing, caching, batching, and choosing the right level of agency — deterministic workflow vs. autonomous agent — for each problem.
• Write production-quality Python with strong testing discipline; contribute to backend services, APIs, and CI/CD pipelines that host agent workloads.
• Partner with product, data science, and design to translate ambiguous business problems into well-scoped agent specifications, success metrics, and rollout plans.
• Stay current on the rapidly evolving agent ecosystem and bring back patterns the team should adopt — or reject — with a clear rationale.

Requirements

• 3+ years of professional software engineering experience shipping production systems, with at least 1 year focused on LLM-powered or agentic applications.
• Strong Python skills, including async programming, type hints, testing, and clean API design. Comfort with Git-based development and modern CI/CD.
• Hands-on experience with one or more agent frameworks (LangGraph, LangChain, OpenAI Agents SDK, Anthropic SDK, CrewAI, AutoGen, Pydantic AI) and provider APIs from at least one of OpenAI, Anthropic, or Google.
• Practical experience with the Model Context Protocol (MCP) or equivalent tool-protocol patterns; ability to design clean tool interfaces and reason about authorization scopes.
• Demonstrated experience building RAG systems, including vector stores (e.g., pgvector, Pinecone, Weaviate), embedding selection, hybrid search, and reranking.
• Working knowledge of agent evaluation: designing evals, building golden sets, running LLM-as-judge, and interpreting results to make ship/no-ship decisions.
• Familiarity with prompt engineering tradecraft and an empirical mindset — preferring measurement over intuition for agent behavior.
• Solid grasp of cloud infrastructure (AWS, GCP, or Azure), containers (Docker), and at least one production runtime — Kubernetes, serverless, or comparable.
• Understanding of LLM security and safety: prompt injection, data exfiltration, output validation, sandboxing, and least-privilege tool access.
• Strong written and verbal communication; ability to write design docs, present trade-offs, and collaborate across product, data, and security functions.

Tech Stack

• AWS
• Azure
• Cloud
• Docker
• Google Cloud Platform
• Kubernetes
• Python

Benefits

• Health, dental, life, and disability insurance
• RRSP with company match
• Paid time off and parental leave
• Teledoc Health services
• Employee recognition and referral bonuses