💼 About This Role

You'll own the evaluation strategy for AI agents at Workato, building both internal frameworks and customer-facing tools. Your work will directly shape how builders debug and improve AI agents at scale. This role offers a dual mandate to define quality metrics and drive adoption across engineering teams.

🎯 What You'll Do

• Define and own the evaluation framework for internal AI agent features.
• Build customer-facing evaluation tools for testing and improving agents.
• Partner with Build Experience PM to integrate evaluation into builder journey.
• Establish metrics for internal agent quality and customer evaluation adoption.
• Spend time with customers to understand struggles in assessing agent performance.

📋 Requirements

• 7+ years in Product Management
• Hands-on experience writing evaluations for AI/ML systems (agents, LLMs, or similar)
• Track record of shipping technical products to both internal and external users
• Experience driving adoption of frameworks or practices across engineering teams

✨ Nice to Have

• Experience with agent architectures, RAG systems, or LLM application development
• Background in ML engineering, solutions architecture, or technical program management
• Familiarity with evaluation frameworks (e.g., human eval pipelines, automated benchmarks)

🎁 Benefits & Perks

• 💰 Competitive compensation with equity
• 🏠 Flexible remote work culture
• 💡 Innovation-driven environment with ownership
• 🎁 Perks and benefits package