Tech Stack

Description

You will design and build data pipelines on GCP, working with structured and unstructured data from various sources. This role involves data ingestion, transformation, and preparation for machine learning models, collaborating with data science teams to ensure data quality and accessibility.

Requirements

• 3+ years SQL with ETL and complex scripting
• 3+ years Python for ETL and data pipelines
• 2+ years GCP experience
• Experience with Spark and distributed systems
• Experience with NoSQL databases (MongoDB, CosmosDB)

Responsibilities

• Data Ingestion from various sources (SQL, Excel, ERP systems)
• Build and maintain ETL/ELT pipelines using Python and SQL
• Data cleaning, wrangling, and quality assurance
• Prepare datasets for supervised and unsupervised ML models
• Manage data pipelines with CI/CD and orchestration tools