Lead Software Platform Engineer, MLOps

Insilico Search Partners is a category-leading Scientific Data and AI Cloud company seeking a Lead Software Platform Engineer to own and scale AI/ML infrastructure as a multi-tenant product. The role involves being the technical design authority for production models, LLMs, and agents, serving scientists at major pharmaceutical companies under compliance obligations.

Responsibilities:

• Own the AI/ML platform architecture — the API surface customers and internal teams build against
• Own the model and prompt lifecycle — registration, versioning, staged promotion, multi-model serving
• Design the inference layer for real-time and batch workloads — routing, caching, GPU capacity planning
• Architect the agentic layer above LLMs — RAG, tool/function calling, MCP-based tooling
• Design multi-tenant security — PII/PHI handling, prompt-injection and tool-abuse defenses
• Build evaluation infrastructure — eval harnesses, regression gates, drift detection
• Own observability and SLI/SLO/SLA definition for production AI systems
• Design for reproducibility and audit trail in a validated pharma environment
• Act as SME and design authority — lead reviews, mentor senior engineers

Requirements:

• 10+ years building distributed, cloud-native systems in production
• Multi-tenant AI/ML infrastructure shipped as an external product — not internal-only pipelines
• Hands-on production LLM experience — RAG, retrieval/embedding design, prompt versioning, tool calling under real traffic
• Expert-level TypeScript and Python
• Production model registry/serving experience — ideally Databricks MLflow
• AI evaluation as a release gate — eval harnesses, regression gates, drift monitoring
• AWS, containerization (Docker), infrastructure-as-code (CloudFormation/CDK)
• Multi-tenant security design experience
• MCP-based tooling and agent runtime experience
• LLM cost/latency optimization, per-tenant cost attribution
• Regulated/validated environment experience — GxP, 21 CFR Part 11, SOC 2
• Life sciences or laboratory data background