Tech Stack

Description

You will build solutions to ingest, transform, and distribute big data for data scientists and products using PySpark, Python, Databricks, and Azure. Take ownership of features through the full SDLC, design scalable data pipelines, implement testing, mentor junior engineers, and collaborate on complex problems.

Requirements

• 3+ years professional data development experience
• 2+ years with Databricks or Hadoop/HDFS
• 2+ years with PySpark/Spark
• 3+ years with Python or Java
• 3+ years with SQL and NoSQL technologies
• Full understanding of ETL and data warehousing concepts
• Experience with pipeline orchestration, CI/CD, and version control
• Strong understanding of Agile (Scrum)
• Bachelor’s Degree (Computer Science, MIS, Math, Business Analytics, or STEM)

Responsibilities

• Design, build, and maintain scalable data pipelines using Kafka, Databricks, and similar tools
• Implement automated unit and integration testing
• Collaborate with architecture and lead engineers on development practices
• Provide mentoring to junior engineers
• Participate in retrospective reviews and estimation for new work
• Collaborate with other engineers to solve complex problems
• Drive improvements in data engineering practices and ways of working