PhD Research Project – Quality Benchmarking for LLM-Based Qualitative Analysis

Nava Insights is building an AI-native qualitative user research platform that uses AI to conduct voice interviews and produce research insights. The project focuses on designing and validating statistical and methodological frameworks to evaluate AI-generated qualitative reasoning, including gold standards, benchmark datasets, scoring methods, model comparisons, and regression testing. It is intended for a PhD student or advanced MSc student working on AI evaluation and qualitative insight quality.

Responsibilities:

• Formalising Nava Insights Gold Standard – Findings (v1.0) from expert human analyses
• Defining evaluation dimensions for insights, themes, and behavioural archetypes
• Designing benchmark datasets based on real qualitative studies
• Developing scoring and aggregation methods that handle subjective judgments
• Comparing:

• Generic LLM-based analysis
• Method- and judgment-driven AI agent systems

• Building a reusable evaluation and regression framework for ongoing model updates