Tech Stack

Description

You will architect and maintain high-throughput data pipelines in Databricks and AWS, optimize performance and cost, enforce data governance, and collaborate with AI Engineers to enable scalable data architecture.

Requirements

• Bachelor's or master's degree in computer science or engineering
• 3+ years of experience in data engineering or related field
• Strong experience in Python development
• Experience with Spark or Scala
• Proficiency in building scalable, distributed data pipelines in a cloud environment

Responsibilities

• Architect, develop, and maintain high-throughput data pipelines in Databricks and AWS
• Ingest, normalize, and enrich large volumes of structured and unstructured data
• Collaborate with AI Engineers and scientists to translate requirements into scalable data architecture
• Optimize pipeline performance and cost using Spark, Delta, and AWS best practices
• Enforce data governance, privacy, and lineage standards with Unity Catalog and fine-grained access controls