Tech Stack

Description

You'll own the architecture for scalable data pipelines using Databricks, Spark, and cloud-native technologies, drive the technical roadmap for Databricks or Palantir Foundry, and partner with data science and analytics teams to align data solutions with strategic objectives. This role involves high-scale engineering, integration leadership, and mentoring team members to establish engineering standards.

Requirements

• Data engineering experience including architecting large-scale cloud-native systems
• Hands-on experience with Databricks or Palantir Foundry in enterprise environments
• Expert-level Python, SQL, Spark, and PySpark
• Strong background with ETL/ELT frameworks and orchestration tools (AWS, GCP, or Azure)
• Proven experience designing distributed systems, optimizing Spark jobs, and building complex data models
• Ability to lead technical decisions, influence stakeholders, and drive engineering standards
• Must be a U.S. Citizen

Responsibilities

• Define and own end-to-end architecture for scalable enterprise-grade data pipelines using Databricks, Spark, and cloud-native technologies
• Drive technical roadmap for Databricks and/or Palantir Foundry including best practices, governance, performance optimization, and tooling standards
• Build, optimize, and maintain distributed data processing workflows supporting large-scale workloads and mission-critical analytics
• Architect and implement data integration patterns across APIs, databases, SaaS systems, and cloud storage environments
• Lead engineering reviews, mentor junior and mid-level engineers, establish coding, observability, and reliability standards
• Partner with data science, analytics, and business teams to define future-state architecture and align data solutions with strategic objectives
• Implement robust data quality frameworks, monitoring, lineage, and observability processes
• Design and deploy cloud-native data solutions using AWS services (S3, Glue, Lambda, Redshift) or equivalent platforms