🛠 Tech Stack

💼 About This Role

You'll join Ironclad's AI Pillar to drive data-driven evaluation and improvement of agentic and ML/LLM systems for contract management. Your work will directly impact features like Smart Import and agentic workflows by analyzing training data, designing feedback loops, and curating high-leverage datasets. This role is critical as AI transforms the legal industry.

🎯 What You'll Do

• Analyze training/evaluation datasets for distribution gaps and inconsistencies.
• Design labeling campaigns, golden datasets, and annotation guidelines.
• Build dashboards tracking model accuracy and product KPIs.
• Investigate failure modes via prompt clustering and error taxonomy.
• Operationalize feedback loops from product telemetry and HITL reviews.

📋 Requirements

• Bachelor's or Master's in quantitative field (Statistics, CS, Data Science, Applied Math).
• 8+ years experience in applied ML/data science, preferably NLP or LLM applications.
• Strong SQL and Python skills with Jupyter, Pandas, experiment tracking.
• Comfortable with ambiguity, large datasets, and cross-functional communication.

✨ Nice to Have

• Experience with prompt analysis, clustering, or user behavior modeling.
• Familiarity with LLM eval techniques, RLHF, or agentic system design.
• Program management experience.

🎁 Benefits & Perks

• 🏥 100% health coverage for employees (medical, dental, vision).
• 👶 Gender-neutral parental leave and family forming support.
• 🏖️ Paid time off - take the time you need.
• 💡 Monthly stipends for wellbeing, hybrid work, and cell phone.
• 🏦 401(k) with employer match through Fidelity.

📨 Hiring Process

Estimated timeline: 2-4 weeks · AI estimate

1. 1Recruiter Phone Screen· 30 min
2. 2Technical Interview· 60 min
3. 3On-site Interview (multiple rounds)· Half day