Staff Engineer, Cloud Platform (Kafka)

Calix is a company focused on delivering world-class products through innovative Cloud Infrastructure solutions. They are seeking a Staff Engineer for their Cloud Platform Engineering team to design and build large-scale Cloud Infrastructure Platforms, specifically managing Kafka Infrastructure to enhance product delivery and performance.

Responsibilities:

• Design, build, and maintain a scalable and reliable Kafka solution using KRaft, Strimzi, Zookeeper, etc
• Implement integration with KRaft and Strimzi for coordination and leader election
• Automate the entire lifecycle of platform components from provisioning through decommissioning
• You’ll help solve problems related to building a global Kafka architecture considering scalability, replication, schema registries and self-service IaC workflows to solve for different use cases such as high traffic telemetry (logs & metrics), business critical events and data processing
• Ensure Observability is an integral part of the infrastructure platforms and provides adequate visibility about their health, utilization, and cost
• Build Cloud native CI/CD pipelines, tools and automation that enables developer autonomy and improves their productivity
• Build tools that predict saturations/failures and take preventive actions through automation
• Collaborate extensively with cross functional teams to understand their requirements; educate them through documentation/training and improve the adoption of the platforms/tools
• Provide technical leadership for your peers and help them deliver the best solutions to the rest of Calix
• As an expert in your area, you will help set the tone for how your team operates. You’ll emphasize modern, rigorous software development practices that emphasize testability, repeatability, and self-service automation
• You’ll conduct code reviews and mentor more junior developers. You’ll openly collaborate with other teams’ leads and help raise the bar of engineering excellence across the entire organization

Requirements:

• Bachelor's degree in Computer Science, Engineering, or related field; or equivalent work experience
• 10+ years of experience in building large scale distributed systems in an always available production environment
• 5+ years of experience in building Infrastructure Platforms and CI/CD pipelines in a major public cloud provider – GCP preferred
• Strong hands on designing integration solutions using Kafka and aware of integration best practices for data streaming solutions
• Strong skills in In-memory applications, Data Integration
• Strong awareness of Kafka ecosystem, KRaft, Strimzi, Zookeeper, Kafka cluster, Kafka broker, producer, consumer, Connectors, different APIs, Kafka Topic etc
• Strong hands on SQL connector, HTTP connector etc
• Strong knowledge on different APIs exposed by Kafka to handle the integration, data steaming and for data management
• Strong awareness on designing deployment architecture for Kafka solutions
• Solid hands on Kafka deployment and configuration
• Experience on production deployment, invoke Kafka components as background processes, configuration, trouble shooting and environment maintenance
• In-depth knowledge of Kubernetes and its ecosystem – Containerd/Docker, Helm/Plulumi, ServiceMesh, Terraform/Terragrunt, Ansible, ArgoCD/Workflow, Tekton, etc
• Hand-on experience with observability platforms/tools like ELK/Fluentd/Fluenbit, Grafana/Prometheus/OpenTelemetry, Cortex/InfluxDB etc
• Experience in Python, Golang or a similar language
• Proven experience in leading major initiatives from requirements, design, and implementation to ongoing lifecycle management
• Experience in coaching and mentoring junior engineers; strong verbal and written communications
• Excellent problem-solving skills, with the ability to troubleshoot complex issues and implement effective solutions
• Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment