Tech Stack

Description

You will design and develop scalable data architectures and pipelines to support data migration to cloud environments. You'll implement ETL processes, ensure data quality, and collaborate with teams to drive successful application migrations for federal clients.

Requirements

• Bachelors degree and a minimum of 5 years experience
• Active Secret Security clearance required
• Experience with cloud-based ETL services (e.g. AWS Glue, Google Cloud Dataflow, Azure Data Factory)
• Experience with Python, SQL, Spark, and PySpark
• Familiarity with data orchestration and data quality processes

Responsibilities

• Design and develop scalable, efficient data architectures to support the migration process
• Create data models and schemas that align with both source and target systems
• Develop and maintain robust data pipelines for data generation, collection, and processing
• Implement real-time and batch processing pipelines as required by the migration strategy
• Design and implement ETL processes to migrate data between source and target systems
• Develop scripts and workflows for data extraction, transformation, and loading
• Develop and maintain data cleansing routines to address data inconsistencies
• Create monitoring systems to track data quality metrics throughout the migration
• Collaborate with the migration team to plan and execute data migration strategies
• Provide expertise on data-related aspects of the migration process
• Troubleshoot and resolve data-related issues during migration
• Optimize data processing and storage for improved performance in cloud environments
• Monitor and tune data pipelines for efficiency
• Implement data security measures to protect sensitive information during migration
• Ensure compliance with data protection regulations and organizational policies
• Develop and maintain data access controls and audit trails
• Create and maintain comprehensive documentation for data architectures and processes
• Work closely with data analysts, cloud engineers, and application developers
• Integrate data solutions with other components of the migrated applications
• Participate in Agile ceremonies (Sprint Planning, Daily Stand-ups, Sprint Reviews, and Retrospectives)