12h ago
Technical Program Manager, Frontier Evals
San Francisco, CA
$207k-$230k / year
full-timemid Hybridai-ml
๐ Tech Stack
๐ผ About This Role
You'll drive frontier evaluation and research programs from concept to analysis, partnering with researchers and engineers. You'll design evals, manage human data campaigns, and build roadmaps. This hybrid IC/PM role offers hands-on technical work in a fast-moving research environment.
๐ฏ What You'll Do
- Manage frontier evaluation projects from research questions to benchmarks
- Design and manage human data campaigns and quality control workflows
- Build roadmaps and operating rhythms for research efforts
- Coordinate across research, engineering, data, product, safety, legal, and vendors
- Do hands-on technical work including prompt iteration and data analysis
๐ Requirements
- Proficiency in Python or SQL for analyzing datasets and automating workflows
- Experience in technical program management, research operations, or similar role
- Understanding of how large language models work, including prompting and evaluation
- Ability to ramp quickly on new domains and turn vague goals into clear plans
โจ Nice to Have
- Experience with model-based evaluation workflows
- Background in safety or frontier model research
๐ Benefits & Perks
- ๐๏ธ Unlimited PTO
- ๐ฐ Equity offered
- ๐ Relocation assistance
- ๐ข Hybrid work with 3 days in office per week
๐จ Hiring Process
Estimated timeline: 2-4 weeks ยท AI estimate
- 1Recruiter Screenยท 30 min
- 2Technical Interviewยท 60 min
- 3Hiring Manager Interviewยท 45 min
0 0 0