🛠 Tech Stack

💼 About This Role

You'll shape the scientific methodology behind pre-training data for large language models, designing ablations and novel algorithms to improve model capabilities. You'll collaborate with engineers and researchers to build scalable data pipelines that directly impact shipped models.

🎯 What You'll Do

• Innovate in data-centric AI: identify and implement novel approaches to data quality and curriculum learning.
• Design and lead rigorous ablation studies across various scales to analyze data composition effects.
• Develop advanced algorithms for scoring and selecting data, such as influence functions or gradient-based matching.
• Collaborate cross-functionally to scale research from prototypes to trillion-token pipelines.

📋 Requirements

• Deep understanding of machine learning theory for foundation model training dynamics and scaling laws.
• Experience designing complex ML experiments related to data composition or curriculum learning.
• Strong Python skills and comfort with PyTorch and deep learning frameworks.
• Willingness to relocate to Heidelberg or travel fortnightly.

✨ Nice to Have

• PhD in machine learning, NLP, or equivalent research experience.
• Contributions to top-tier venues like NeurIPS, ICML, ICLR, or ACL on data curation or LLM pre-training.
• Experience training foundation models from scratch and diagnosing data-induced pathologies.

🎁 Benefits & Perks

• 🏖️ 30 days paid vacation
• 🏋️ Wellhub fitness & wellness membership
• 🧠 nilo.health mental health support
• 💰 Substantially subsidized company pension plan
• 🚆 Subsidized Germany-wide transportation ticket

📨 Hiring Process

Estimated timeline: 2-4 weeks · AI estimate

1. 1Recruiter Call· 30 min
2. 2Technical Interview· 60 min
3. 3On-site or Virtual Final Round· 90 min