🛠 Tech Stack

💼 About This Role

You'll drive open-source integrations for the LanceDB multimodal AI database, working at the intersection of high-performance computing, big data, and open-source systems. You'll collaborate with a world-class team on cutting-edge AI infrastructure.

🎯 What You'll Do

• Drive open-source community integrations with Spark, Hive, Presto, Trino, Ray
• Design and maintain efficient distributed Lance dataset operations
• Build indices for predicate pushdown in Spark, Ray, or Trino
• Work on Lance format in Rust and improve data processing infrastructure

📋 Requirements

• 10+ years building high-performance databases or big data systems
• Deep understanding of open-source Big Data systems internals (e.g., Spark, Flink, Iceberg, Trino)
• Strong experience with high-performance computing in Java or Scala
• Experience with Rust (or willingness to learn)

✨ Nice to Have

• Contributor or PMC member in Apache or other large open-source projects
• Experience with Apache Arrow, DataFusion, Parquet
• Track record of driving large features in distributed systems

🎁 Benefits & Perks

• 📈 Competitive compensation, equity, and benefits
• 🏖️ Remote-first team with flexible hours
• 📚 Generous learning budget
• 🌐 Support for open-source contributions