Tech Stack

Description

You will design evaluation frameworks for GenAI features and internal AI agents, run online experiments to measure impact, define success metrics, and build automated evaluation systems to improve engineering efficiency and code intelligence at Roblox.

Requirements

• PhD or Master’s in Statistics, Economics, Computer Science, Applied Math, Physics, Engineering or related quantitative field.
• 5+ years experience in data science, analytics, or quantitative role.
• Strong proficiency in SQL (Hive/Spark) and scripting languages (Python or R).
• Solid grounding in experimentation, causal inference, and statistical analysis.
• Familiarity with GenAI models and safety/quality evaluation methods.

Responsibilities

• Design and operationalize rigorous evaluation systems for GenAI features or internal AI Agents.
• Conduct online experiments (A/B tests) and causal inference to quantify impact.
• Define leading/lagging indicators for GenAI safety, user satisfaction, engineering productivity, and code health.
• Research and apply state-of-the-art methodologies to build reproducible evaluation tooling and agentic workflows.
• Develop dashboards and reporting frameworks to reveal trends and translate data into recommendations for leadership.