The UVA VECRemote
Senior Bash Software Engineer (AI Evaluation)
United States of America
Contract
3 weeks ago
Visa Sponsor
Key skills
BashShell scriptingUnix/Linux command-line environmentsSoftware engineeringSystems engineeringAlgorithmic problem solvingScript debuggingSystem processesFile systemsNetworking basicsAutomation workflowsLLM usage in codingEvaluation frameworksShellAICI/CDCommunication
About this role
The UVA VEC is focused on enhancing the reliability of conversational systems through a structured AI evaluation initiative. They are seeking a Senior Bash Software Engineer to evaluate AI-generated Bash scripts, validate outputs, and provide feedback to improve system performance.
Responsibilities:
- Evaluate AI-generated responses to scripting and systems engineering tasks
- Execute and validate Bash scripts to ensure correctness and reliability
- Identify logical errors, inefficiencies, and edge case failures in shell workflows
- Annotate outputs with structured feedback on strengths and weaknesses
- Assess script readability, maintainability, and operational safety
- Perform fact-checking using reliable technical documentation and references
- Apply standardized evaluation frameworks and scoring criteria
- Ensure outputs align with expected system behavior and engineering standards
Requirements:
- 5+ years of professional experience in software engineering, systems engineering, or related fields
- Strong expertise in Bash and shell scripting
- Deep familiarity with Unix/Linux command-line environments
- Ability to solve medium to hard algorithmic and scripting problems independently
- Experience writing, executing, and debugging production-level scripts
- Understanding of system processes, file systems, networking basics, and automation workflows
- High attention to detail in reviewing technical reasoning and outputs
- Fluent English communication skills
- Experience using LLMs in coding or scripting workflows and understanding their limitations
- Ability to follow structured evaluation frameworks and guidelines
- Bachelor's degree or higher in Computer Science or related discipline
- Experience contributing to open-source projects with accepted contributions
- Familiarity with additional programming or scripting languages
- Experience in model evaluation, RLHF, or data annotation
- Background in DevOps, infrastructure automation, or CI/CD systems
- Experience reviewing scripts in production environments
- Ability to explain complex system behaviors and scripting concepts clearly