Key skills
AWSCloudGoogle Cloud PlatformPythonTypeScriptAILLMOpenAIVector DatabaseGCPGoogle CloudCommunication
About this role
Role Overview
- Build and maintain an internal LLM gateway that handles routing, fallbacks, and rate limiting
- Create reusable components for common AI patterns (RAG, function calling, streaming responses)
- Develop SDKs or libraries that simplify AI integration for application developers
- Implement prompt management systems and evaluation frameworks
- Design and deploy vector database infrastructure for semantic search
- Create monitoring dashboards for AI usage, costs, and performance metrics
- Write technical documentation and best practice guides for AI implementation
- Evaluate and integrate new AI services and models as they become available
- Build testing frameworks for prompt regression and model output validation
Requirements
- 5+ years of software engineering experience with strong focus on platform/infrastructure development
- Hands-on experience with LLM APIs (OpenAI, Anthropic, etc.) and AI orchestration frameworks (OpenAI Agents SDK, Google ADK)
- Proficiency in Python or TypeScript
- Knowledge of vector databases, embedding models, and RAG architectures
- Experience building internal developer tools, SDKs, or platform services
- Understanding of API design, authentication, and rate limiting
- Familiarity with cloud platforms (AWS, GCP) and containerization
- Experience with observability, monitoring, and cost management for AI systems
- Strong documentation and communication skills
Tech Stack
- AWS
- Cloud
- Google Cloud Platform
- Python
- TypeScript
Benefits
- Comprehensive Medical, Dental and Vision plans (ButterflyMX covers 80% of the cost) starting day 1
- 401(k) plan with a match
- 10 paid holidays, 20 vacation days, 5 sick days, 3 floating holidays
- Basic Life and Accidental Death and Dismemberment Insurance (ButterflyMX covers 100% of the cost)
- Short and Long Term Disability (ButterflyMX covers 100% of the cost)
- Paid Family Leave
- Employee Assistance Program
- Quarterly self-care stipends
- Access to optional benefits including pre-tax flexible healthcare spending accounts (FSA and HSA), Dependent Care FSA, and Commuter Benefits, as well as optional Supplemental Life, AD&D, Hospital Indemnity, Legal, Accident, Critical Illness, Pet, and Personal Liability Insurance
- And more!