Tech Stack

Description

You will challenge advanced language models on tasks like hallucination detection, factual consistency, prompt-injection resistance, and bias auditing, documenting failure modes to improve model reliability. On a typical day, you will design test plans, verify factual accuracy, and build evaluation rubrics, working with automation and red-teaming to track quality improvements.

Requirements

• Bachelor's, master's, or PhD in CS, data science, or related field
• Experience with QA for ML/AI systems and safety/red-teaming
• Hands-on work with LLM eval tooling (e.g., OpenAI Evals)
• Test automation frameworks (e.g., PyTest, Python/SQL)
• Clear communication and ability to document findings

Responsibilities

• Design and run test plans and regression suites
• Evaluate model outputs for hallucination, bias, and factual accuracy
• Build clear rubrics and pass/fail criteria
• Capture reproducible error traces with root-cause hypotheses
• Partner on adversarial red-teaming and automation