Key skills
PythonAILLMRisk Management
About this role
Role Overview
- Design and implement model evaluation frameworks across AI products
- Develop methodologies for:
- Bias and fairness testing
- Hallucination and reliability assessment
- Robustness and stress testing
- Safety benchmarking
- Evaluate training data governance practices
- Review RAG systems for retrieval accuracy and exposure risks
- Establish measurable risk metrics across AI deployments
- Align evaluation outputs with:
- NIST AI Risk Management Framework
- ISO 27701 privacy requirements
- Enterprise governance standards
- Produce structured, executive-ready documentation
- Partner with product and engineering teams to integrate risk mitigation strategies
Requirements
- Experience designing model evaluation frameworks
- Familiarity with bias detection methodologies
- Understanding of hallucination testing and reliability measurement
- Experience stress-testing LLM-based systems
- Strong Python and experimentation capabilities
- Knowledge of NIST AI RMF
- Familiarity with privacy-by-design principles
- Experience operating within ISO 27001 / SOC 2 environments
- Understanding of enterprise AI risk posture expectations
- Ability to translate model risk into business impact
- Strong documentation skills
- Comfortable presenting findings to executive audiences
- Systems-thinking mindset
Tech Stack
- Python
Benefits
- Comprehensive Private Medical Coverage
- Support for Mental Health Expenses
- Life Insurance Options
- Attractive Compensation Package