🛠 Tech Stack

💼 About This Role

You'll lead annotation pipelines and model evaluation for a scaled vector embeddings ecosystem at CreatorIQ, an influencer marketing platform. Your work will directly impact cost efficiency and product outcomes by ensuring the right model is deployed for each task.

🎯 What You'll Do

• Architect human-in-the-loop annotation workflows using Label Studio
• Create golden sets and evaluation criteria to benchmark models
• Implement model-as-a-judge and PII scrubbing in production pipelines
• Integrate measurement loops into AWS/GCP infrastructure

📋 Requirements

• Annotation experience with tools like Label Studio or Scale AI
• Python proficiency for glue code and API integrations
• Experience with model monitoring and evaluation in production
• Cloud infrastructure fluency in AWS (Sagemaker, S3) and GCP (Vertex AI)

✨ Nice to Have

• Experience with smaller model optimization for cost efficiency
• DevOps collaboration skills for deployment of solutions
• Familiarity with generative AI and LLM evaluation

🎁 Benefits & Perks

• 🍔 Surprise meal stipends for work-from-home
• 🏖️ 26 days vacation plus floating holidays and wellness allowance
• 🏥 Medical insurance, life insurance, and business travel insurance
• 📈 Stock options as part of equity-sharing program
• 📱 Comprehensive perks including cell/internet stipend, home office setup, and tuition reimbursement

📨 Hiring Process

Estimated timeline: 2-4 weeks · AI estimate

1. 1Recruiter Call· 30 min
2. 2Technical Interview· 60 min
3. 3Team Interview· 60 min