Key skills
PythonJavaGoAIGenerative AIRAGVector DatabaseAWSAzureGoogle CloudKubernetesK8sEKSAKSGKEHelmElasticsearch
About this role
Cloudera is a leading data partner for top companies, empowering them to transform complex data into actionable insights. As a Staff Engineer on the Cloudera Context Search Team, you will lead the technical vision for high-performance search infrastructure and optimize integration with the Cloudera Data Platform.
Responsibilities:
- Drive the multi-quarter technical roadmap and design large-scale OpenSearch clusters capable of handling petabytes of data with low-latency indexing and query performance
- Deeply integrate OpenSearch with CDP components (e.g., Apache Iceberg, SDX, and Ozone) to provide a unified search experience across the data lakehouse
- Lead efforts to optimize JVM settings, shard allocation strategies, and query DSL to ensure maximum throughput and stability
- Oversee the development of Kubernetes Operators and Helm charts for automated deployment, scaling, and self-healing of search services
- Define and champion best practices for security (RBAC, TLS), observability, and enterprise-grade reliability
- Mentor senior and junior engineers on complex technical designs and foster a culture of continuous improvement across the organization
- Act as a primary liaison and influencer within the upstream OpenSearch community, aligning their roadmap with product strategy
Requirements:
- Bachelor's degree in Computer Science or equivalent, and 6+ years of experience; OR Master's degree and 4-6 years of experience; OR PhD and 2-4 years of experience
- 6+ years of experience working with OpenSearch or Elasticsearch in production environments at scale
- Deep understanding of consensus algorithms, CAP theorem, replication, and sharding
- Mastery of Java (for core development) and proficiency in Go or Python for automation and tooling
- Extensive experience with Kubernetes (K8s) and container orchestration
- Hands-on experience deploying search workloads on AWS (EKS/AOSS), Azure (AKS), or Google Cloud (GKE)
- Experience with Lucene internals (segment merging, bitsets, and codecs)
- Knowledge of Vector Database capabilities within OpenSearch for Generative AI (RAG) use cases
- History of contributing to open-source projects (Apache Software Foundation or OpenSearch Project)