Design and implement scalable replication services across HDFS, Hive, HBase, Apache Iceberg, and other big data technologies
Lead complex data migration initiatives between on-premises clusters and cloud environments including AWS S3 and Azure ADLS Gen2
Build robust APIs and microservices for the Replication Manager platform
Design fault-tolerant, petabyte-scale distributed systems with comprehensive monitoring, alerting, and observability capabilities
Ensure data security and governance compliance during movement operations
Drive technical decisions for new features and evaluate emerging technologies
Partner with CDP, SRE, and field engineering teams to integrate replication capabilities
Guide junior engineers on best practices and conduct code reviews

8+ years in software engineering with strong proficiency in Java, Scala, or Python
Deep hands-on experience with the Apache Hadoop ecosystem (HDFS, Hive, HBase, YARN)
Solid experience with modern data formats including Apache Iceberg, Delta Lake, and Hive tables with ACID support
Practical experience across AWS, Azure, and GCP storage services
Working knowledge of containerization tools like Docker and Kubernetes
Proven ability to architect large-scale distributed systems
Familiarity with security protocols and data governance frameworks
Well-versed in agile SDLC, CI/CD pipelines, automated testing, Git-based code review workflows, and observability tooling including Prometheus, Grafana, and the ELK stack

Staff Software Engineer – Replication Manager

Key skills