Senior Big Data Engineer - Cloudera Hadoop to AWS Migration

Randstad Digital Americas is seeking a senior-level Big Data Engineer with hands-on experience migrating enterprise Cloudera Hadoop workloads to AWS. The role involves leading migration planning, target architecture design, application replatforming, validation, and technical documentation.

Responsibilities:

• Design and develop scalable, maintainable software and data-processing solutions
• Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines
• Develop and execute migration strategies from on-premises Hadoop to AWS
• Design AWS architectures using EMR, Glue, Airflow, Lambda, and related services
• Oversee data transfer, application re-platforming, testing, and validation
• Ensure solutions meet performance, scalability, security, and reliability requirements
• Troubleshoot software, platform, and data-pipeline issues
• Create technical documentation, migration plans, and operational runbooks
• Collaborate with engineering, DevOps, QA, product, and business stakeholders
• Mentor team members on software engineering and AWS big data practices

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field
• 10+ years of experience with enterprise application design, Hadoop, and big data platforms
• Hands-on experience with Cloudera Hadoop, HDFS, Hive, Spark, Oozie, and Python
• Proven experience migrating or modernizing Hadoop workloads on AWS
• Strong experience with Amazon EMR, AWS Glue, Airflow, Lambda, and cloud data architecture
• Proficiency with Bash or similar shell scripting languages
• Experience with Terraform or other Infrastructure-as-Code tools
• Strong understanding of cloud security, governance, compliance, and data protection
• Experience with XML and JSON data formats
• Familiarity with monitoring and logging tools
• 5+ years of experience with tools such as Bitbucket and Artifactory
• Strong communication, documentation, troubleshooting, and mentoring skills
• AWS certification, such as AWS Certified Solutions Architect or a related data certification
• Experience working with healthcare claims data, providers, members, and related healthcare entities
• Previous experience leading large-scale Hadoop-to-AWS migration initiatives