Content Engineer | MonGenAI

TEKsystems is a leading provider of business and technology services, and they are seeking a Senior Content Engineer to join their team. In this role, you will be responsible for building, evaluating, and optimizing AI-driven content systems while ensuring high-quality outcomes through collaboration with program managers, engineers, and researchers.

Responsibilities:

• Partner with the Program Manager to design, iterate, and maintain end-to-end content evaluation guidelines
• Create structured artifacts such as specs, decision frameworks, mappings, and changelogs to document guideline evolution
• Ensure guidelines are actionable, internally consistent, and aligned with human evaluation standards
• Identify and resolve cases where guidelines are ambiguous, inconsistent, or fundamentally unlabelable
• Design, test, and refine system prompts to guide AI models toward consistent, high-quality, human-aligned outputs
• Apply structured experimentation to evaluate prompt effectiveness and document tradeoffs
• Collaborate cross-functionally to version, validate, and improve prompts over time
• Analyze model evaluation results using appropriate metrics, focusing on distributions, prevalence, slices, and uncertainty, not just averages
• Identify patterns, failure modes, and quality risks across datasets and model outputs
• Translate analysis into clear, actionable recommendations for model behavior and process improvements
• Support supervised fine-tuning efforts by preparing, annotating, and quality-controlling large-scale text datasets
• Define and maintain gold set lifecycles, annotation specs, and QC plans
• Provide editorial oversight to ensure labeling consistency, accuracy, and alignment with evaluation goals
• Define, document, and maintain known and emerging failure states for content evaluation and generation systems
• Proactively surface risks related to consistency, nuance, safety, and edge cases
• Collaborate with partners to address root causes and implement mitigation strategies
• Identify opportunities to improve content pipelines, workflows, and quality controls
• Think in systems: inputs, contracts, versioning, monitoring, and outputs
• Balance tradeoffs such as speed vs. quality and consistency vs. nuance when proposing improvements

Requirements:

• Experience in developing and maintaining content evaluation guidelines
• Ability to create structured artifacts such as specs, decision frameworks, mappings, and changelogs
• Experience in designing, testing, and refining system prompts for AI models
• Ability to apply structured experimentation to evaluate prompt effectiveness
• Experience in analyzing model evaluation results using appropriate metrics
• Ability to identify patterns, failure modes, and quality risks across datasets and model outputs
• Experience in preparing, annotating, and quality-controlling large-scale text datasets
• Ability to define and maintain gold set lifecycles, annotation specs, and QC plans
• Experience in defining, documenting, and maintaining known and emerging failure states for content evaluation and generation systems
• Ability to identify opportunities to improve content pipelines, workflows, and quality controls
• Experience in balancing tradeoffs such as speed vs. quality and consistency vs. nuance