Staff Software Engineer

Levi, Ray & Shoup, Inc. (LRS) is seeking a Staff Software Engineer to design and build scalable platform infrastructure for AI-driven applications. This role involves technical leadership in building internal platforms and developer tools while collaborating with cross-functional teams to drive innovation.

Responsibilities:

• Design and build core platform components, including:
• LLM provider abstractions and SDKs
• Structured output validation systems
• Streaming and asynchronous processing systems
• Token management frameworks
• Architect scalable, event-driven systems using modern cloud infrastructure
• Build systems that support production-grade LLM use cases (e.g., OpenAI, Anthropic)
• Develop evaluation frameworks for model quality, including datasets, scoring, and regression detection
• Design and implement prompt lifecycle management (versioning, testing, deployment)
• Implement guardrails, PII detection/redaction, and audit logging systems
• Build privacy-focused observability and monitoring solutions
• Ensure responsible AI usage and compliance best practices
• Build SDKs and platform tools used across multiple teams
• Create documentation, reference implementations, and onboarding materials
• Partner with product and engineering teams to understand needs and drive adoption
• Influence architecture, engineering standards, and best practices
• Lead by example through hands-on development and code reviews
• Promote testing, type safety, and observability across systems

Requirements:

• 8+ years of software engineering experience
• Strong background in platform engineering, distributed systems, or developer tools
• Production experience with LLM/AI systems
• Expertise in TypeScript and backend development (Node.js/NestJS preferred)
• Experience with event-driven architecture and asynchronous processing
• Hands-on experience with AWS cloud services
• Strong understanding of observability (metrics, tracing, alerting)
• Knowledge of AI safety, PII handling, and auditability
• Proven ability to lead technically and influence across teams
• Strong communication and documentation skills
• Experience building SDKs or internal platforms used across organizations
• Familiarity with: Prisma ORM, PostgreSQL
• DynamoDB, Redis
• Docker and containerized environments
• Experience with: Prompt engineering and LLM evaluation tools
• Streaming systems for real-time AI applications
• Multi-tenant platform design
• Ports-and-adapters or similar architectural patterns
• Exposure to modern AI tooling ecosystems (e.g., Vercel AI SDK, OpenAI Agents SDK)
• Contributions to AI or open-source projects