Tech Stack

Description

You will build the next generation of ML Ecosystem Tooling for model inference at Roblox, supporting billions of requests per day. You'll set technical strategy, optimize inference stacks, and partner across organizations to make ML tooling delightful to use.

Requirements

• 4+ years professional experience with system design
• Experience building distributed systems for real-time ML inference serving at millions of QPS
• Experience debugging infrastructure-level performance issues for low latency, high throughput
• Bachelor's degree in Computer Science or related field
• Familiarity with ML inference frameworks like Triton Inference Server, TensorRT, KServe

Responsibilities

• Set technical strategy and oversee development of high-scale inference infrastructure
• Optimize performance from model to infrastructure level
• Stay abreast of industry trends in ML and infrastructure
• Bootstrap and maintain ML Platform components: Serving Layer, Metadata Store, Model Registry, Pipeline Orchestrator
• Partner across organizations to build tooling, interfaces, and visualizations