Key skills
Technical SupportQuality Assurance (QA)Software Development Engineer in Test (SDET)TroubleshootingRoot Cause AnalysisAPI TestingPostmanLog AnalysisApplication DebuggingSQLAI/LLM SystemsPrompt EngineeringRetrieval-Augmented Generation (RAG)Agent OrchestrationAI-powered Development ToolsAILLMClaudeRAGAzureAzure DevOpsSalesforceCommunication
About this role
Ascensus is the leading independent technology and service platform powering savings plans across America. The AI Platform Software Engineer supports and scales adoption of the enterprise AI platform by troubleshooting AI behavior, identifying systemic gaps, and driving continuous improvement in platform performance and usage.
Responsibilities:
- Investigate and resolve issues reported by users of Agent Assist
- Reproduce issues and trace failures across:
- LLM responses and prompts
- Agent workflows and orchestration
- Tool/MCP calls and backend integrations (e.g., Salesforce and internal enterprise applications)
- Analyze logs, traces, and telemetry (e.g., Langfuse, NewRelic logs)
- Clearly document findings and partner with engineering on resolution leveraging Azure DevOps
- Identify whether issues stem from prompting, retrieval (RAG), tool selection, or backend systems
- Propose improvements to agents, skills, prompts, and workflows
- Use AI-powered tools (e.g., Cursor) to accelerate debugging, analysis, and solutioning
- Contribute to continuous improvement of system reliability and response quality
- Train internal users on how to effectively use Agent Assist
- Run targeted demos, onboarding sessions, and office hours
- Act as a trusted advisor to client-facing associates
- Drive adoption by helping users understand both capabilities and limitations
- Partner closely with engineering, product, and AI teams
- Translate user issues into clear, actionable tickets and improvements
- Identify recurring patterns and escalate systemic issues
- Responsible for protecting, securing, and proper handling of all confidential data held by Ascensus to ensure against unauthorized access, improper transmission, and/or unapproved disclosure of information that could result in harm to Ascensus or our clients
Requirements:
- Up to 3 years' experience in a technical support, QA, SDET, or engineering-adjacent role (including internships, co-ops, or academic projects)
- Bachelor's degree in Computer Science, Computer Information Systems, Business Information Systems, or a related technical field
- Demonstrated ability to troubleshoot and debug technical issues, focusing on root cause analysis
- Foundational experience or exposure to API testing (Postman or similar)
- Reading logs and debugging application behavior
- Basic SQL or data inspection
- Exposure to or strong interest in AI/LLM-based systems (prompting, chatbots, RAG, or agents) through coursework, projects, or hands-on experimentation
- Experience using or willingness to learn AI-powered development tools (e.g., Cursor, Claude Code)
- Strong communication skills with the ability to translate technical issues into clear explanations for non-technical users
- Demonstrated curiosity and ability to learn quickly in a fast-evolving technical environment
- Qualified candidates must complete a video interview assessment after applying as the next step, to be completed within 7 days of receiving the link. How to prepare: Set aside 40-45 minutes for the self-guided assessment that includes games and questions. Find a quiet place to record and be camera-ready. You'll need a smartphone, tablet, or desktop computer with your camera and microphone enabled. You'll answer questions to share your skills and experience and bring your personality to the interview. This step accelerates the interview process, moving qualified candidates to hiring manager interview fast
- Experience supporting or building AI/agent-based applications
- Familiarity with Retrieval-Augmented Generation (RAG)
- Tool/agent orchestration
- Prompt engineering