Lead the design and implementation of our Kafka ecosystem, including configuration, deployment, and optimization for reliability, performance and scalability
Collaborate with stakeholders to gather requirements and define Kafka use cases and architectures that align with business objectives
Design and implement Kafka integration with other data systems and pipelines
Implement security measures to protect Kafka clusters and data, including encryption, authentication, and authorization
Establish monitoring, alerting, and logging mechanisms to ensure the health and performance of Kafka clusters, and troubleshoot issues as they arise
Provide guidance and best practices to development teams on Kafka usage, including producers, consumers, and stream processing applications
Collaborate with development teams to define Kafka topics, partitions, and configurations based on application requirements
Automate deployment, configuration, and monitoring tasks using tools like Terraform, Ansible, Puppet, or Chef
Stay up-to-date with the latest Kafka developments and best practices, and make recommendations for continuous improvement.

Degree in Computer Science, a related field, or equivalent work experience
Proven experience as a Kafka Engineer or similar role, with hands-on experience deploying and managing Kafka clusters in production Cloud environments
Deep understanding of Kafka architecture, including brokers, topics, partitions, replication, and consumer groups
Extensive experience with Kafka administration, monitoring, and troubleshooting
Experience with Kafka security features, such as SSL/TLS encryption, Kerberos authentication, and ACLs
Experience with Kafka integration patterns and technologies, such as Kafka Connect and other Kafka ecosystem tools
Proficiency in programming and scripting languages, such as Java, Scala, Python, or Shell scripting
Excellent problem-solving skills and attention to detail
Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.

Senior Engineer - Kafka

Key skills