Confluent - Staff Software Engineer - Kafka

IBM is hiring a Staff Software Engineer to advance the Realtime Context Engine, including its multi-tenant, highly available compute infrastructure and query execution and background materialization components. The role focuses on scalable query strategies and reliable platform infrastructure for Confluent Cloud.

Responsibilities:

• As a Staff Software Engineer on the team, you will focus on advancing key layers of the Realtime Context Engine project, including its multi-tenant, highly available compute infrastructure, and the core components responsible for query execution and background materialization. You will tackle challenges such as developing strategies for efficient execution and at massive scale across all of Confluent Cloud, ensuring high availability and scalability
• Query Engine Development: Design and implement innovative strategies for query execution, planning, and caching at massive scale
• Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers computational tasks for Lightning tables and the realtime context engine. Ensure that the platform is optimized for scalability, reliability, and resilience, in alignment with organizational objectives
• Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage features and infrastructure with the broader organization

Requirements:

• Expertise in C++ and/or Rust
• 10+ years of demonstrated professional work experience building database technologies
• Experience in building and operating large-scale database systems
• Distributed Systems domain experience
• Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels
• Master's Degree
• Familiarity with or has experience contributing to the following open-source technologies: Apache Iceberg, Velox, Substrait
• Experience/knowledge with public clouds (AWS, Azure or GCP)
• Computer Science Degree with Specialization in Database Technologies