Key skills
ApacheHadoopHBaseHDFSKafkaSparkYarnZookeeperApache KafkayarnCI/CD
About this role
Role Overview
- Monitor, troubleshoot, and resolve production job failures (Oozie, Spark)
- Manage Apache Kafka clusters, including Kafka Connect, Schema Registry, and topic management
- Maintain HBase, Sqoop, and ZooKeeper for reliability
- Perform deep-dive analysis on incident reports
- Tune NiFi flows, Kafka producers/consumers, and Spark/Hive queries
- Collaborate with DevOps to implement CI/CD processes
Requirements
- Minimum 3 to 5 Yrs of experience
- Apache NiFi Administration: Install, configure, and maintain Apache NiFi/NiFi Registry
- Kafka Management: Manage Apache Kafka clusters
- Hadoop Ecosystem Support: Monitor HDFS, YARN, Hive, and Spark jobs
- Root Cause Analysis (RCA): Perform deep-dive analysis on incident reports
- Performance Tuning: Tune NiFi flows, Kafka producers/consumers, and Spark/Hive queries
- Deployment and Upgrades: Collaborate with DevOps to implement CI/CD processes
Tech Stack
- Apache
- Hadoop
- HBase
- HDFS
- Kafka
- Spark
- Yarn
- Zookeeper
Benefits
- Health insurance
- Retirement plans
- Paid time off
- Flexible work arrangements
- Professional development