Senior Machine Learning Engineer (IntelliJ AI)

JetBrains is a company passionate about developing effective developer tools since 2000. They are seeking a Senior Machine Learning Engineer to enhance AI assistance across JetBrains IDEs, focusing on developing and evaluating ML systems, collaborating with product teams, and mentoring junior engineers.

Responsibilities:

• Design and drive evaluation frameworks for AI features, including metrics, experiments, and agent trace analysis
• Diagnose model performance issues (e.g. prompt drift, context mismatches, and latency/quality trade-offs) and translate findings into actionable improvements
• Experiment with contexts and lightweight models to continuously develop our ML system
• Act as the ML liaison for product teams across JetBrains, adapting and scaling AI capabilities in JetBrains IDEs to their needs
• Build and maintain small helper models (e.g. re-rankers, classifiers, embedding models) to support domain-specific tasks
• Collaborate with colleagues in ML, product, engineering, and analytic teams to deliver improvements and monitor their impact in production
• Stay up to date with research in the fields of LLMs, agents, and evaluation, bringing best practices into our workflows
• Mentor junior engineers and help shape team culture, processes, and tooling around experimentation and evaluation