About this role

We are hiring a Data Scientist with strong Applied AI experience to develop, test, and improve intelligent applications powered by Large Language Models. The role focuses on building reliable AI solutions, optimizing retrieval-based architectures, and establishing systematic evaluation approaches for LLM-generated outputs. Candidates should have strong Python and analytical capabilities, along with practical experience in model benchmarking, automated quality assessment, and human-assisted evaluation. Familiarity with modern AI orchestration frameworks and cloud-based ML platforms is expected.

Roles & Responsibilities

Develop and enhance intelligent applications using Large Language Models, advanced prompting strategies, Python-based data processing, and enterprise AI APIs.
Design retrieval-based AI solutions by implementing embedding generation, vector search, document indexing, context retrieval, and response optimization techniques.
Establish model assessment methodologies to measure the consistency, factual accuracy, relevance, and overall performance of GenAI applications.
Implement automated evaluation pipelines using LLM-as-a-Judge, human feedback mechanisms, and frameworks such as RAGAS, DeepEval, TruLens, and LangSmith.
Analyze model performance using quantitative measures, including precision, recall, F1-score, retrieval quality, groundedness, and response relevance.
Develop multi-step AI workflows and intelligent agent solutions using LangGraph, LangChain, and other orchestration technologies.
Perform exploratory data analysis, benchmarking, and evaluation reporting using Python, Pandas, SQL, Excel, and relevant ML experimentation tools.
Integrate AI monitoring, experiment tracking, evaluation logging, and model performance analysis using MLflow, Databricks, Arize AI, and AWS services.

Data Scientist – Generative AI & RAG Evaluation

Key skills

About this role