Tech Stack

Description

You will design and develop data pipelines for ingesting and egressing research data, optimize their performance, and implement data governance best practices to support AI tooling integration.

Requirements

• 5+ years of professional experience in data engineering or similar role
• Strong experience with MongoDB, SQL, and Databricks
• Experience developing and refactoring ETLs
• Solid understanding of data architecture principles and data modeling
• Fluency with Git and unit testing using TDD

Responsibilities

• Design and develop data egress pipelines for downstream research data consumption
• Design and develop data ingress pipelines for composed dashboards of research products
• Optimize data pipeline performance, ensuring high availability, reliability, and data quality
• Implement data governance best practices including lineage, cataloging, auditing, and access controls
• Ensure data structures and pathways support integration with AI tooling