Key skills

PythonBackend Services and APIsDistributed SystemsAWS, Azure, or Google Cloud PlatformDockerKubernetesGitCI/CDInfrastructure as Code (IaC)Large Language Models (LLMs)Retrieval-Augmented Generation (RAG)Prompt EngineeringAIGenerative AILLMLarge Language ModelsRAGAgenticAWSAzureGoogle CloudPerformance Optimization

About this role

LTS is seeking an AI Platform and Harness Engineer to build and maintain infrastructure, tooling, and evaluation frameworks for enterprise AI solutions. The role focuses on developing reusable AI harnesses, LLMOps pipelines, observability systems, backend services, and secure cloud-native deployments for LLM, agent, RAG, and Generative AI applications.

Responsibilities:

Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications
Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance
Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance
Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics
Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement
Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking
Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity
Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations
Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications
Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures
Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment
Apply security, governance, and Responsible AI controls throughout the AI development lifecycle
Evaluate emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to improve engineering productivity
Troubleshoot production AI issues and continuously improve platform reliability, scalability, security, and user experience
Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices