CareSourceRemote
AI Systems Analyst III
United States
Full Time
1 week ago
$94,100 - $164,800 USD
Visa Sponsor
Key skills
AzureCloudAILLMLlamaRAGAgenticAnalyticsDatabricks
About this role
Role Overview
- Analyze incoming AI use cases to determine technical feasibility, data availability, and appropriate risk tiering (green/yellow/red).
- Perform detailed data lineage and quality assessments to ensure training/RAG datasets meet governance standards for accuracy and PII/PHI protection.
- Maintain the 'AI Agent Registry & Catalog,' documenting agent capabilities, API dependencies, and ownership within the Azure APIM and Mesh architecture.
- Draft technical requirements and 'Definition of Ready' artifacts for the Platform Engineering team, ensuring a smooth handoff from CoE to Engineering.
- Support the AI Engineering Committee (AIEC) by documenting architectural fit and identifying potential technical debt in proposed solutions.
- Monitor and aggregate telemetry data on token usage, cost, and error rates to support the 'Value Analyst' in ROI reporting.
- Assist in the creation of 'Data Dictionaries' and 'Knowledge Graphs' required to ground RAG (Retrieval Augmented Generation) pipelines.
- Validate that yellow layer automations (UiPath, Databricks) utilize approved MCP connectors and do not bypass API gateways.
- Collaborate with Data Governance to tag and classify datasets specifically approved for LLM training or fine-tuning.
- Track the lifecycle of AI models from 'Pilot' to 'Production' to 'Retirement' in the enterprise inventory system.
- Support the configuration of 'Model Routers' by analyzing performance benchmarks across different LLMs (GPT-4 vs. Llama) for specific tasks.
- Create process flow diagrams for 'Agentic Workflows' to visualize how multiple agents interact and hand off tasks.
- Perform any other job related duties as requested.
Requirements
- Bachelor's degree in Information Systems, Computer Science, Data Analytics, or related field required
- Equivalent years of relevant work experience may be accepted in lieu of required education
- Five (5) years in Systems Analysis, Data Analysis, or Technical Product Ownership required
- Experience documenting technical requirements for data-intensive applications, API integrations, or cloud platforms required
- Background in Healthcare Payer data (Claims, Member, Provider) is preferred.
Tech Stack
- Azure
- Cloud
Benefits
- Base compensation and potential for bonuses tied to company and individual performance
- Substantial and comprehensive total rewards package