Tech Stack

Description

You will own and evolve systems for large-scale web data collection, designing and maintaining production-grade scraping and ingestion infrastructure. You'll build scalable, observable, and resilient systems that enable multiple teams to reliably add and operate data sources, working closely with data engineers and product teams to scale data pipelines.

Requirements

• 5+ years in production-grade backend/data/platform systems with web scraping experience
• Solid understanding of networking fundamentals (TLS/SSL, timeouts, failure modes)
• Hands-on with browser automation tools; proficiency in CSS selectors and XPath
• Experience with robust retry logic, error handling, testing, and observability
• Proficiency with SQL and NoSQL databases; experience on AWS including VPC networking

Responsibilities

• Build a scalable web data acquisition platform used across teams
• Enable other teams to ingest data safely and efficiently
• Make cost-reliability-performance tradeoffs in architecture decisions
• Create shared abstractions and tooling for scrapers in production
• Own system reliability with retries, error handling, and failure recovery