Applied AI/ML Engineer

Certum Legal Solutions is building an AI-native platform for legal services. The Applied AI/ML Engineer owns the platform’s AI/ML intelligence layer, developing production LLM systems for document intelligence, task agents, quality assurance, and operational workflows in a HIPAA-regulated environment.

Responsibilities:

• Own all AI/ML system design and implementation on the platform
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing
• Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation
• Design the agent orchestration framework powering task agents, QA agents, and operational agents
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
• Partner with the Lead Backend Engineer on model abstraction layer design and API integration
• Partner with domain experts on extraction schema design and workflow logic

Requirements:

• 5+ years of software engineering experience with at least 2–3 years focused on production LLM application development
• Expert Python; strong API integration skills; experience deploying AI systems at production scale
• Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them
• Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
• RAG architecture: vector databases, embedding models, retrieval optimization
• Prompt engineering discipline: systematic evaluation, regression testing, version control for prompts
• Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)
• Document and text extraction from unstructured sources (OCR post-processing, information extraction from clinical or legal documents)
• Conversational AI or voice AI systems (NLU, dialog management, conversation evaluation)
• Legal or healthcare AI applications
• Multi-agent orchestration patterns
• Experience evaluating and scoring LLM outputs at scale