Tech Stack

Description

You will design, implement, and optimize end-to-end data pipelines for scraping and processing structured and unstructured data using Google Cloud Platform. You'll conduct ad hoc web scraping and data collection to support research and intelligence initiatives, and prepare data for further analysis including cleaning, transformation, anonymization, and masking. You'll collaborate with ML engineers, other data engineers, and software developers to deliver actionable insights and functional tools.

Requirements

• Degree (or equivalent work experience) in Computer Science, Engineering, Information Science, Data Science or a related field (graduate degree preferred)
• 2+ years of professional experience in data engineering or a closely related field
• Ability to communicate complex technical ideas clearly to non-technical audiences
• Proficiency in Python, SQL
• Experience with web scraping/crawling (e.g., Beautiful Soup, Selenium, Scrapy)
• Experience with Google Cloud Platform (or similar), including storage and database services (e.g., Cloud Storage, CloudSQL, Cloud Spanner) and workflow orchestration (e.g., Cloud Composer/Airflow, Cloud Run, Pub/Sub)
• Experience building and managing data pipelines, especially for text data
• Comfort working in fast-moving, high-impact environments, such as startups, AI research labs, or security-focused teams

Responsibilities

• Design, implement, and optimize end-to-end data pipelines for scraping and processing structured and unstructured data
• Conduct ad hoc web scraping and data collection to support research and intelligence initiatives
• Prepare data for further analysis, including data cleaning, transformation, anonymization, and masking
• Contribute to the development of internal and external APIs, following best practices
• Collaborate with ML engineers, other data engineers, and software developers to deliver actionable insights and functional tools
• Drive other critical initiatives