Role Overview
The AI Architect will lead the architecture, design, and production rollout of enterprise-grade AI solutions, with a focus on Generative AI (GenAI), Retrieval-Augmented Generation (RAG), and agentic AI systems. The role translates business needs into secure, scalable, reliable, cost-effective architectures and establishes reusable architecture patterns, engineering standards, and governance practices. The architect partners with product, data, platform, security, engineering, and business leaders to take AI capabilities—from conversational assistants and document intelligence to workflow automation and decision support—from experimentation into production.

Key Responsibilities

Define enterprise AI architecture strategy, reference architectures, technology standards, and a pragmatic roadmap aligned to business outcomes.
Design end-to-end AI solutions spanning data ingestion, model access, retrieval, orchestration, APIs, user experience, deployment, monitoring, and governance.
Architect production GenAI, RAG, and agentic workflows with appropriate controls for quality, latency, availability, scalability, and cost.
Assess and select foundation models, embedding models, vector/search technologies, agent frameworks, cloud services, and deployment patterns; document trade-offs and decisions.
Establish reusable components, architecture patterns, design reviews, engineering guidelines, and platform capabilities that accelerate delivery across teams.
Lead technical discovery, prototypes, architecture spikes, and production readiness reviews; guide teams through implementation and operational handover.
Partner with security, privacy, legal, risk, and compliance stakeholders to implement responsible AI, data protection, access controls, auditability, and risk-based safeguards.
Define evaluation and observability practices for AI systems, including quality, groundedness, safety, drift, latency, usage, and cost metrics.
Mentor engineers and technical leads; communicate architecture decisions and risks to executive, product, and engineering stakeholders.
Mandatory Skills & Experience
1. Enterprise AI & GenAI Architecture
Demonstrated experience designing and deploying production-grade GenAI applications such as conversational assistants, summarization, document intelligence, knowledge assistants, and decision support systems.
Strong understanding of LLM behavior and constraints: tokenization, context windows, prompting, structured outputs, tool/function calling, model selection, and failure modes.
Experience designing inference and serving patterns for latency, throughput, and cost, including streaming,batching where appropriate, caching, asynchronous processing, and response chunking.
Ability to evaluate build-versus-buy, hosted-versus-self-hosted, and model/platform trade-offs.
2. Retrieval-Augmented Generation (RAG) & Knowledge Systems
Hands-on architecture experience with ingestion, parsing, metadata enrichment, chunking, embeddings, indexing, retrieval, reranking, context assembly, and grounded generation.
Knowledge of dense retrieval, BM25, hybrid search, metadata filtering, semantic ranking, query rewriting, multi-hop retrieval, and retrieval evaluation.

Experience with vector databases/search platforms such as pgvector, Pinecone, Weaviate, Milvus, Qdrant, Chroma, Elasticsearch/OpenSearch, or cloud-native equivalents.
Design for enterprise knowledge sources, document permissions, freshness/synchronization, provenance, feedback loops, and measurable retrieval quality.
3. Agentic AI & Workflow Architectures
Experience architecting tool-using agents and multi-agent/workflow systems, with clear boundaries for planning, execution, delegation, state, memory, and human approval.
Familiarity with frameworks such as LangGraph, LangChain, Microsoft Agent Framework / AutoGen, Semantic Kernel, and CrewAI; select frameworks based on use case rather than framework preference.
Design reliable orchestration using state machines, durable workflows, event-driven patterns, retries, timeouts, idempotency, and human-in-the-loop controls.
Knowledge of tool/function calling, MCP (Model Context Protocol), API integration, agent-to-agent communication patterns, and secure tool authorization.
Understand planning-and-execution, reflection/critique patterns, memory approaches, and the limits of autonomous behavior; use bounded autonomy and deterministic controls for critical actions.
4. Cloud, Platform & Infrastructure (AWS, Azure, Google Cloud Platform)
Architecture experience on at least one major cloud platform (AWS, Microsoft Azure, or Google Cloud); working familiarity with the other platforms and ability to map equivalent services.
AWS: Amazon Bedrock, SageMaker, S3, Lambda, EKS, IAM, CloudWatch, and relevant data/search services.
Azure: Azure AI Foundry, Azure Machine Learning, Azure OpenAI where applicable, Azure Kubernetes Service
(AKS), Functions, Entra ID, Azure Monitor, and data/search services.
Google Cloud Platform: Vertex AI, Gemini, BigQuery, Cloud Storage, GKE, Cloud Run, IAM, and Vertex AI Search or relevant retrieval services.
Experience with cloud networking, identity and access management, secrets, encryption, private connectivity, environment separation, resilience, and cost controls.
Containerization and deployment experience with Docker and Kubernetes or managed equivalents; infrastructure- as-code familiarity (Terraform, CloudFormation, Bicep, or similar).
5. AI/ML Engineering, APIs & Data
Strong Python skills and practical experience with production software engineering practices; familiarity with REST APIs, event-driven integration, and service contracts.
Knowledge of ML lifecycle concepts: data preparation, training/fine-tuning, model registry, deployment, versioning, monitoring, and rollback.
Experience with data pipelines and enterprise data architectures, including SQL, object storage, lakehouse/warehouse patterns, and batch/stream processing.
Familiarity with PyTorch or TensorFlow, Hugging Face ecosystem, and model-serving options such as managed endpoints, vLLM, or equivalent technologies.
Knowledge of CI/CD, automated testing, Git-based workflows, observability, and production operations.
6. Security, Governance, Evaluation & Responsible AI
Design defenses against prompt injection, jailbreaks, sensitive-data exposure, insecure tool use, data exfiltration, and unsafe model outputs.
Apply least privilege, tenant/data isolation, identity-aware retrieval, encryption, secrets management, retention controls, and audit logging.
Define AI evaluation strategies for task success, groundedness, relevance, factuality, safety, bias, robustness, and regression testing; combine automated and human evaluation.
Familiarity with LLM/agent observability and tracing tools such as Langfuse, LangSmith, OpenTelemetry, cloud- native monitoring, or equivalents.
Working knowledge of responsible AI and risk frameworks such as NIST AI RMF, ISO/IEC 42001, and applicable privacy/security obligations; experience in regulated domains is beneficial.

Good-to-Have Skills (Differentiators)
LLM customization and optimization: fine-tuning, PEFT, LoRA/QLoRA, quantization, distillation, prompt tuning, and inference optimization.
Model benchmarking and evaluation experience; awareness of task-specific metrics and limitations of generic metrics such as BLEU, ROUGE, or perplexity for conversational/grounded systems.
Experience with multimodal AI (text, image, audio, video), speech/voice interfaces, vision pipelines, or real-time applications.
Experience with federated/multi-source RAG, enterprise content integrations (CRM, document management, wikis, intranet), and permission-aware indexing.
Exposure to open and commercial models/providers such as OpenAI, Anthropic, Google Gemini, Meta Llama, Mistral, Cohere, and other suitable model families.
Experience with agent memory architectures, durable state, workflow engines, and production agent analytics.
Experience in healthcare, financial services, or another regulated enterprise environment; familiarity with HIPAA, HITRUST, or other sector-specific controls where applicable.
Qualifications & Experience
Typically 10+ years of software, data, platform, or AI/ML engineering experience, including substantial architecture ownership; equivalent experience may be considered.
Typically 3+ years designing or delivering AI/ML solutions, with recent hands-on experience in GenAI, RAG, or agentic systems.
Experience leading architecture across multiple teams or enterprise initiatives, including design reviews, technical decisions, and production outcomes.
Bachelor’s or master’s degree in Computer Science, Engineering, Data Science, or a related field, or equivalent practical experience.
Expected Outcomes / Success Measures
AI solutions move from proof of concept to production with clear business value and operational ownership.
Architecture patterns are secure, reusable, scalable, observable, and maintainable.
AI quality, groundedness, safety, latency, availability, and cost are measured and continuously improved.
Teams adopt consistent architecture standards and deliver with reduced duplication and improved engineering effectiveness.

Fervorly

Open Positions (1)

Engineering

AI Architect – Generative AI & Agentic AI

Meet the Hiring Team