Key skills
PythonLarge Language Models (LLM)Retrieval-Augmented Generation (RAG)AI evaluation pipelinesPerformance metrics implementationBackend systemsAPIsCloud environments - AWSCloud environments - GCPLogging toolsDebugging toolsTypeScriptAIMLLLMRAGMLOpsLangGraphStatistical AnalysisAWSGCP
About this role
Lattice is a people success platform focused on enhancing employee development and engagement. They are seeking a Software Engineer specializing in AI to contribute to AI evaluation pipelines and improve the quality of AI systems, ensuring high-quality user experiences.
Responsibilities:
- Contribute to AI evaluation pipelines, including offline evals, production tracing, and feedback systems
- Implement and maintain performance metrics (e.g., response quality, task success, reliability) using established frameworks
- Help create and maintain evaluation datasets and test cases to identify regressions
- Analyze results and propose incremental improvements to model and agent quality
- Contribute to AI system components such as RAG pipelines, retrieval systems, and multi-step workflows within existing architectures
- Write clean, maintainable Python code that integrates with LLM providers and internal services
- Support improvements to system reliability, observability, and performance in production
- Deliver well-scoped projects with guidance from more senior engineers
- Break down tasks, make steady progress, and be proactive in unblocking yourself by asking for help when needed
- Contribute to team excellence through code reviews, documentation, and knowledge sharing
- Collaborate with cross-functional partners to ship user-facing features
Requirements:
- 2–5 years of professional software engineering experience
- Experience contributing to production systems as part of a team
- Exposure to AI/ML systems with a strong interest in LLM-powered products
- Experience debugging systems, working with data, and iterating on performance
- Proficiency in Python or a similar language
- Strong understanding of LLM concepts (prompting, RAG, evaluation)
- Familiarity with backend systems, APIs, and cloud environments (e.g., AWS, GCP)
- Exposure to logging, monitoring, or debugging tools
- Interest in learning tools like LangGraph, vector databases, and evaluation platforms
- Strong ownership: you reliably deliver high-quality work on well defined tasks, on time and communicate progress clearly
- Learning mindset: you actively seek feedback and improve quickly
- Pragmatic and product-minded: you focus on solving problems effectively and not perfectly
- Collaborative: you contribute to team discussions and uphold engineering best practices
- Growth-oriented: you actively seek feedback and expand your skills in AI engineering
- Hands-on experience with LLMs, prompt iteration, or MLOps
- Familiarity with vector databases or retrieval systems
- Exposure to experimentation, metrics, or basic statistical analysis
- Familiarity with TypeScript