🛠 Tech Stack

💼 About This Role

You'll work in our pre-training team building distributed training and inference for Large Language Models. You'll have access to thousands of GPUs to verify changes and focus on software development best practices and code architecture. You'll help train the best foundational models for source code generation.

🎯 What You'll Do

• Propose and evaluate innovations in training development experience and reliability
• Enhance and maintain training and inference codebases
• Write high-quality Python, Cython, C/C++ code and perform refactorings
• Improve CI/CD pipelines

📋 Requirements

• Understanding of Large Language Models (LLM)
• Basic knowledge of Transformers
• Strong engineering background with Python, PyTorch, or Jax
• Experience with C/C++ and CI/CD

✨ Nice to Have

• Experience with distributed training systems
• Knowledge of deep learning fundamentals
• Linux proficiency

🎁 Benefits & Perks

• 🏖️ Fully remote work and flexible hours
• 🌴 37 days/year of vacation and holidays
• 💊 Health insurance allowance for you and dependents
• 💻 Company-provided equipment
• 📚 Always-be-learning and home office allowances

📨 Hiring Process

Intro call with a Founding Engineer, technical interviews, team fit call, final interview.