Tech Stack

Description

You will design and build all data, ML pipelines, data tools, and cloud infrastructure to transform massive, fragmented data into a format that supports processes and standards, directly empowering business stakeholders to gain comprehensive visibility and drive strategic impact.

Requirements

• 5+ years of hands-on experience designing and maintaining big data pipelines
• Proficiency in Python, Scala, Java, or Go and expertise in SQL and NoSQL databases
• Experience with software engineering best practices: testing, code reviews, CI/CD
• Experience developing data pipelines and maintaining data lakes on AWS
• Familiarity with Kafka and workflow orchestration like Airflow

Responsibilities

• Design and build data, ML pipelines, and supporting infrastructure
• Build and optimize an AWS-based Data Lake using cloud architecture best practices
• Lead end-to-end data projects from design through production monitoring
• Implement optimal ETL/ELT patterns for challenging data integration problems
• Solve integration challenges from structured and unstructured data