Remote DevOps Engineer (Kafka/Kong)

Pennant Solutions is seeking a skilled and experienced Senior Kafka/Kong Administrator to join their IT operations team. The role involves deploying, configuring, maintaining, and monitoring Kafka clusters and Kong gateways to ensure high availability and optimal performance for real-time platforms.

Responsibilities:

• Deploy and configure Kafka clusters following best practices for scalability, security, and performance
• Collaborate with cross-functional teams to gather requirements and design Kafka infrastructure to meet data streaming needs
• Manage topics, partitions, replication factors, and broker configurations to ensure efficient data distribution and fault tolerance
• Implement monitoring and alerting solutions to proactively identify and address performance bottlenecks, resource constraints, and anomalies
• Conduct regular performance testing, load testing, and capacity planning to ensure clusters can handle anticipated workloads
• Design and implement high-availability strategies, including failover mechanisms and data replication across multiple data centers or cloud regions
• Develop and maintain disaster recovery plans and procedures to minimize data loss and downtime in the event of failures
• Implement and manage security measures such as encryption, authentication, and authorization to ensure data privacy and compliance with industry standards
• Stay updated on security vulnerabilities and patches, applying necessary updates to maintain a secure Kafka environment
• Diagnose and resolve Kafka-related issues, including performance degradation, data loss, and connectivity problems
• Collaborate with development and data engineering teams to troubleshoot consumer/producer application integration with Kafka
• Maintain thorough documentation of Kafka configurations, deployment processes, and troubleshooting procedures
• Provide training and knowledge sharing sessions to junior team members and other stakeholders
• Collaborate with cross-functional teams, including data engineers, developers, and system administrators, to ensure smooth integration of Kafka into our data ecosystem
• Communicate effectively with stakeholders to provide updates on Kafka performance, maintenance, and improvements
• Hands on experience installing and configuring Kong API Gateway as well as integration with existing API frameworks
• Describe and configure dataplane security using RBAC, Roles, Workspaces and Teams
• Configure upstreams and load balancing
• Create Services, Routes, and Consumers
• Working knowledge of Kong Ingress with Kubernetes/Dockers
• Excellent knowledge of API design standards, patterns, and best practices especially with OpenAPI 2.0, REST, JSON, XML
• Distinguish between downstream and upstream networking issues
• Collecting troubleshooting/debug information
• Using log files and health and traffic metrics
• Deep understanding of JWTs and their usage
• Use Kong Vitals to monitor Kong Gateway’s health and performance
• Pull metrics using Vitals API

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent work experience)
• 5+ years of hands-on experience in administering Apache Kafka clusters in production environments
• 5+ years proficiency in Kafka/Kong architecture, installation, configuration, and tuning
• Strong understanding of Kafka internals, including topics, partitions, replication, and consumer/producer APIs
• 5+ years of experience with Kafka ecosystem tools such as Kafka Connect, Kafka Streams, and Schema Registry
• Knowledge of Linux systems and shell scripting
• Strong DevOps background
• 5+ years of familiarity with cloud platforms (e.g., AWS, Azure, GCP) and containerization (e.g., Docker, Kubernetes)
• Excellent problem-solving skills and the ability to troubleshoot complex issues efficiently
• Strong communication skills and the ability to work collaboratively in a team environment
• Certification as a Confluent Certified Administrator for Apache Kafka (CCAAK)
• Preferred certification in Kong Gateway Certified Associate (KGCA)