🛠 Tech Stack

💼 About This Role

You'll design and build the evaluation stack for Exa's AI-powered search engine, creating benchmarks and pipelines that measure search quality in an LLM world. Your work will directly shape the direction of search research and influence what the team optimizes for. You'll collaborate with ML researchers and engineers to build the most comprehensive eval suite for billions of documents.

🎯 What You'll Do

• Design evaluation frameworks that probe search limits
• Build scalable eval pipelines tracking regressions and drift
• Create golden datasets, synthetic benchmarks, and agentic tasks
• Partner with ML researchers and engineers on feedback loops

📋 Requirements

• Hands-on ML experience training or evaluating models
• Strong engineering fundamentals in Python or Rust
• Experience with distributed pipelines and GPU/cluster jobs
• Ability to dive into data and design creative measurement strategies

✨ Nice to Have

• Experience with embeddings or LLMs
• Background in search or information retrieval

🎁 Benefits & Perks

• 🏖️ Unlimited PTO
• 🩺 Premium medical, dental, vision
• 👶 Fertility benefits
• 💪 Monthly wellness stipend