Software Development Engineer II

Valiant Harbor International is a CVE Service-Disabled Veteran Owned Small Business specializing in technical, programmatic, acquisition, compliance, and financial services for government agencies. They are seeking a Software Development Engineer II to support the development of the General Research Assistant and Content Engine (GRACE), focusing on building agentic AI systems and backend features to enhance research and decision-making processes for ARPA-H.

Responsibilities:

• Assist in building agentic AI Systems:
• Implement and iterate on GRACE's agentic workflows: tool-calling, multi-step reasoning, planning, memory, and agent-to-agent (A2A) communication patterns at the application layer
• Build and maintain MCP (Model Context Protocol) client-side integrations (e.g., how GRACE agents discover, invoke, and compose tools)
• Implement tool definitions, input/output schemas, error handling, retry logic, and result formatting for GRACE's growing tool library
• Contribute to multi-agent orchestration patterns that are reliable and debuggable in production
• Build LLM-powered features:
• Implement LLM orchestration logic (prompt construction, context management, model selection, etc.), and buld/maintain RAG pipeline components (query formulation, result ranking, citation grounding, and hallucination mitigation)
• Implement and iterate prompt engineering patterns and system prompts that drive GRACE's quality and consistency across model families
• Contribute to context window budget management (e.g., truncation, summarization, and pagination logic)
• Build LLM evaluation components (grounding assessment, regression tests, safety checks, and quality metrics)
• Write prompts and pipelines with token economics in mind (i.e., cost-per-query )
• Build backend features and contribute to Infrastructure:
• Build secure, well-tested features end-to-end from application logic through to the API
• Implement integrations with internal and external data sources and APIs, including Dimensions, Google Search, Slack, SharePoint, and LLM provider APIs
• Contribute to monitoring, logging, and distributed tracings to diagnose failure and regressions early
• Implement fallback, retry, and graceful degradation patterns for AI service dependencies
• Write production-quality code: readable, tested, reviewed, and documented
• Work within GRACE's Microsoft Azure infrastructure: Azure Functions, Azure API Management, Azure Container Apps, and Azure OpenAI Service
• Contribute to CI/CD pipelines, deployment automation, and release processes
• Work with containerization tools and infrastructure as code; understand the environment your code runs in
• Contribute to application-level SLOs: tool call success rates, response quality, and latency from the user's perspective
• Collaborate with internal/external partners:
• Participate actively in design reviews, sprint planning, and retrospectives; ask good questions and push back when something does not add up
• Communicate technical decisions clearly to both engineers and non-engineers
• Work closely with the PM, researcher, designer, and senior engineers to translate ambiguous requirements into clear, actionable implementations
• Bring genuine curiosity and empathy to every feature; understand who is using what you build and why it matters to them

Requirements:

• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field, or equivalent practical experience
• 3+ years of professional software engineering experience building and operating production systems
• Proven experience in high-velocity environments where you contributed to shipping real products end-to-end
• Strong proficiency in Python and at least one other backend language; familiarity with modern backend frameworks and async patterns
• Solid understanding of algorithms, data structures, distributed systems, and software design patterns
• Experience building and operating systems on major cloud platforms (AWS, GCP, or Azure)
• Experience with containerization (Docker) and working within CI/CD pipelines
• Clear, direct communicator who gives and receives feedback well, works with empathy, and makes the people around them better
• Experience with Microsoft Azure (Azure Functions, API Management, Container Apps, or Azure OpenAI Service)
• Hands-on experience building features on top of LLMs in production: tool-calling, RAG, multi-step reasoning, and context management
• Familiarity with A2A (Agent-to-Agent) communication patterns and multi-agent orchestration frameworks
• Familiarity with MCP at the client/consumer layer (i.e., how agents discover and invoke tools via MCP)
• Working knowledge of prompt engineering and LLM behavior across model families (e.g., why Claude and GPT respond differently to the same prompt)
• Experience with LLM evaluation, grounding assessment, or regression testing for AI-powered systems
• Experience in startup or early-stage environments; comfort with ambiguity and rapid iterations