🛠 Tech Stack

💼 About This Role

You'll build the data and evaluation backbone for AI-native developer workflows at Guild.ai, establishing the company's truth layer from product instrumentation to evaluation frameworks for autonomous systems. You will define KPIs, create AI evaluation systems, and directly shape product direction. This role offers a unique opportunity to build a measurement culture from scratch.

🎯 What You'll Do

• Define product KPIs and quality metrics for AI behaviors
• Establish event taxonomy and instrumentation standards
• Design offline and online evaluation for agentic workflows
• Run A/B tests and produce actionable analyses

📋 Requirements

• Statistics and experimentation foundations
• Fluency in SQL and Python
• Experience building analytics in fast-moving environments
• Ability to translate ambiguous questions into clear recommendations

✨ Nice to Have

• Experience evaluating LLMs or agentic systems
• Familiarity with developer tools and observability
• Experience with modern data tooling (dbt, warehouses)

🎁 Benefits & Perks

• 📈 Significant equity in a venture-backed startup
• 🏥 Comprehensive Health Benefits (Medical, Dental, Vision)
• 🏖️ Flexible PTO