LLM Platform Engineer

AllSpice is hiring an LLM Platform Engineer to build the tooling, evaluation systems, and data platforms that support its Generative AI team. The role focuses on AI product evaluation and observability, internal applications, reliable data pipelines, integrations, and improving engineering workflows.

Responsibilities:

• Partner with engineers and product managers to discover workflow bottlenecks, prioritize the highest-impact problems, and own solutions end-to-end—from requirements and architecture to rollout and iteration
• Own the evaluation and observability layer for our AI products, including automated and human review workflows, regression detection, and comparisons across models, prompts, and pipelines. Make quality, cost, latency, usage, and customer outcomes visible and actionable
• Build usable internal applications and self-service tools that help engineers and product managers inspect AI results, investigate failures, and provide feedback. Where useful, extend these capabilities into customer-facing experiences
• Build and maintain scripts, reliable data pipelines, and integrations that connect internal systems, third-party services, LLMs, and Model Context Protocol (MCP) tools, with attention to data quality, schema evolution, and customer data privacy
• Write clear documentation and communicate findings to both technical and non-technical audiences, including leadership and customers
• Champion code quality, testing, and reliability across the tooling stack through reviews, mentorship, and thoughtful design