Senior Software Engineer, LLMs

Role Overview

• Design and build user-facing ML features that harness LLMs and generative AI to unlock new product capabilities
• Partner with product, design, and ML research to prototype and deliver high-impact, ML-powered experiences
• Own the technical architecture and implementation strategy for applied ML systems
• balancing latency, observability, and iteration speed
• Build scalable services and APIs that bring model outputs to users in trustworthy and intuitive ways
• Collaborate across platform, infra, and legal/compliance teams to ensure ML deployments meet standards for safety, fairness, and performance
• Establish and evangelize best practices for prompt design, model evaluation, and experimentation across the org

Requirements

• 4+ years of software engineering experience, with 2+ years working directly on ML-driven products or intelligent systems
• Proven ability to lead complex initiatives across engineering, product, and research stakeholders
• Strong backend development skills (e.g., Python with FastAPI or Flask), plus experience with cloud-native tooling (e.g., Kubernetes, Docker, Terraform)
• Experience integrating LLMs or ML models into production systems, including APIs and user-facing applications
• Excellent communication skills and a collaborative, product-minded approach
• Ability to think rigorously about system design, latency tradeoffs, and user impact when working with ML features
• Experience shipping GenAI or LLM-powered features using frameworks like LangChain, LlamaIndex, or OpenAI APIs
• Familiarity with retrieval-augmented generation (RAG), vector search (e.g., FAISS, Pinecone), and real-time inference patterns
• Proficiency in full-stack development, including front-end work with React or similar frameworks
• Strong intuition for prompt engineering, model testing, and evaluation methodologies
• Experience navigating complex requirements around explainability, user trust, or compliance in ML applications
• Track record of influencing architecture or product direction at a team or org level

Tech Stack

• Cloud
• Docker
• Flask
• Kubernetes
• Python
• React
• Terraform

Benefits

• Competitive compensation, including base pay, bonus opportunities, and annual equity grants that vest quarterly
• Generous 401(k) plan with Upstart matching $2 for every $1 contributed, up to $15,000 per year
• Employee Stock Purchase Plan (ESPP) with discounted stock purchase options for eligible employees
• Affordable medical, dental, and vision coverage, with multiple plan options
• Upstart covers 90% to 100% of the cost depending on the plans you choose
• Health Savings Account contributions from Upstart for eligible plans
• Income protection benefits, including company-paid Basic Life, AD&D, and Short
• and Long-Term Disability coverage, with options to purchase supplemental coverage
• Paid time off, sick and safe time, and company holidays
• Paid family and parental leave to support caregiving and major life moments
• Family-centered benefits through Carrot and Cleo, supporting fertility, parenthood, and caregiving
• Employee Assistance Program (EAP) offering mental health support and life-centered resources
• Financial wellness resources, including access to financial planning tools and a financial concierge service
• Annual wellness allowance to support your physical and emotional well-being and personal development, based on what matters most to you
• Annual productivity allowance to invest in relevant tools and resources you need to do your best work, no matter where you work from
• Connection and community through team events and onsites, all-company updates, and employee resource groups (ERGs)
• Onsite perks, including catered lunches and fully stocked micro-kitchens when working from one of our four offices, located in the Bay Area, Austin, Columbus, and New York City (opening Summer 2026!).