Senior AI Application Engineer - Backend - Azure OpenAI, RAG, Azure AI Search/Elasticsearch

JOB DESCRIPTION

POSITION: Senior Application AI Backend Engineer with Azure OpenAI, RAG, Azure AI Search/Elasticsearch
DURATION: Long term
Location: New York, NY - Onsite

Job Description

• Design, build, & maintain backend services, APIs, & application features for firmwide products & AI-enabled workflows.
• Develop AI application capabilities such as enterprise search, summarization, information extraction, document generation, and conversational experiences using large and small language models.
• Design and integrate secure connections among applications, AI services, enterprise data, and business systems through well-defined APIs, identity controls, and access policies.
• Build retrieval solutions that make relevant enterprise information available to AI applications, including content ingestion, metadata processing, embeddings, indexing, and vector, semantic, or hybrid search.
• Apply sound software engineering practices, including modular design, code review, automated testing, CI/CD, observability, error handling, and performance tuning.
• Evaluate AI features for quality, groundedness, reliability, security, latency, and cost; use evaluation results and user feedback to improve them over time
• Partner with product, application, data, security, and business teams to translate needs into technical designs and explain implementation choices and trade-offs.
• Own solutions through design, development, deployment, and production support; contribute to technical standards and help other engineers grow.
• Strong experience developing backend systems, APIs, and production applications, preferably with C#/.NET. Experience with Python, Java, JavaScript, or TypeScript is also relevant.
• Experience with REST APIs, distributed systems, cloud applications, databases, & integration with third-party services.
• Experience delivering applications that use generative AI or LLMs; familiarity with retrieval-augmented generation (RAG), structured outputs, prompt design, and tool or API calling.
• Experience with Microsoft Agent Framework or a comparable agent framework, and with designing AI-based applications, web experiences, chat interfaces, dashboards, or internal business platforms.
• Familiarity with Azure AI Foundry, Azure OpenAI, and Azure cloud services; experience with AWS Bedrock, Google Vertex AI, or comparable platforms is also useful.
• Experience with Elasticsearch and/or Azure AI Search, embeddings, vector search, semantic and hybrid retrieval, and the indexing and evaluation of enterprise content.
• Knowledge of LLM and SLM capabilities, model selection, inference, and application patterns; experience using enterprise platforms such as ChatGPT Enterprise is preferred.
• Familiarity with production monitoring, automated evaluation, application security, data privacy, & responsible AI practices.
• 10+ years of professional software engineering or related experience, including substantial work on backend services, APIs, applications, or distributed systems.
• At least 2 years of experience building or integrating Gen AI or LLM-based solutions in applications or business systems.
• Programming skills in C#/.NET preferred; experience with Python, Java, JavaScript, or TypeScript is also valuable.
• Design & integrate APIs & services & deliver software from development through deployment & production support.
• Automated testing, CI/CD, cloud deployment, logging, monitoring, and troubleshooting production applications.