Staff Software Engineer, LLMs

Upstart is a leading AI lending marketplace focused on reducing the cost and complexity of borrowing. The Staff Software Engineer will work within the Applied LLM team to design and build user-facing ML features that leverage large language models and generative AI, collaborating with various teams to enhance product capabilities and ensure safety and performance standards.

Responsibilities:

• Design and build user-facing ML features that harness LLMs and generative AI to unlock new product capabilities
• Partner with product, design, and ML research to prototype and deliver high-impact, ML-powered experiences
• Own the technical architecture and implementation strategy for applied ML systems - balancing latency, observability, and iteration speed
• Build scalable services and APIs that bring model outputs to users in trustworthy and intuitive ways
• Collaborate across platform, infra, and legal/compliance teams to ensure ML deployments meet standards for safety, fairness, and performance
• Establish and evangelize best practices for prompt design, model evaluation, and experimentation across the org

Requirements:

• 6+ years of software engineering experience, with 2+ years working directly on ML-driven products or intelligent systems
• Proven ability to lead complex initiatives across engineering, product, and research stakeholders
• Strong backend development skills (e.g., Python with FastAPI or Flask), plus experience with cloud-native tooling (e.g., Kubernetes, Docker, Terraform)
• Experience integrating LLMs or ML models into production systems, including APIs and user-facing applications
• Excellent communication skills and a collaborative, product-minded approach
• Ability to think rigorously about system design, latency tradeoffs, and user impact when working with ML features
• Experience shipping GenAI or LLM-powered features using frameworks like LangChain, LlamaIndex, or OpenAI APIs
• Familiarity with retrieval-augmented generation (RAG), vector search (e.g., FAISS, Pinecone), and real-time inference patterns
• Proficiency in full-stack development, including front-end work with React or similar frameworks
• Strong intuition for prompt engineering, model testing, and evaluation methodologies
• Experience navigating complex requirements around explainability, user trust, or compliance in ML applications
• Track record of influencing architecture or product direction at a team or org level