Tech Stack

Description

You will collaborate with clients and internal teams to integrate healthcare data into a modern cloud-based data platform, building and optimizing ETL pipelines using Databricks and Snowflake. Your work ensures accurate data ingestion and transformation, supporting health plans in making better care and cost decisions.

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 2+ years of hands-on experience as a Data Engineer working with distributed data processing systems in modern cloud environments.
• Proficiency in Python and SQL for data processing and analysis.
• Experience building and managing ETL pipelines using Databricks workflows or similar tools.
• Familiarity with structured and semi-structured data and processing with PySpark.
• Basic knowledge of cloud services such as AWS (S3, SQS) or equivalent.
• Understanding of version control and CI/CD tools (e.g., GitLab).
• Ability to optimize queries and troubleshoot performance issues in SQL or SparkSQL.
• Exposure to data warehousing technologies like Snowflake or similar platforms.

Responsibilities

• Collaborate with clients and implementation teams to understand data distribution requirements.
• Develop and maintain data pipelines using Databricks and Snowflake to ensure data accuracy and integrity.
• Build ETL workflows and optimize data processes for efficiency and scalability.
• Translate business requirements into technical solutions in partnership with cross-functional teams.
• Implement best practices for data security and compliance with healthcare regulations.
• Document processes, workflows, and technical specifications to support ongoing improvements.