Tech Stack

Description

In this role, you will enhance the Python-based ingestion framework to support ingesting metadata from systems like Snowflake, Redshift, and Kafka, build connectors for modern data and ML stacks, enable the framework to run in a cloud-native environment, leverage AI for advanced capabilities, and lead and mentor a team of junior engineers.

Requirements

• Possess strong CS fundamentals with a degree in Computer Science or equivalent
• Proficient in Python with hands-on experience building and delivering features end-to-end
• 5+ years of experience designing, developing, and shipping distributed backend systems
• Have a solid understanding of RESTful API design, algorithmic complexity, and distributed systems
• Adaptable and thrive in fast-paced, evolving environments
• Experience working in Data ecosystems (like Snowflake, Databricks) and multiple clouds is a strong plus
• Experience in leading a small team (pod) is a strong plus

Responsibilities

• Enhance the Python-based ingestion framework to support ingesting usage statistics, lineage, and operational metadata from systems like Snowflake, Redshift, Kafka, more
• Build connectors for major systems in the modern data and ML stacks
• Enable the ingestion framework to run in a cloud native environment
• Leverage AI for advanced connector capabilities and metadata ingestion inference
• Lead and mentor a team of junior engineers