Diverse LynxRemote
Kafka Full stack Engineer
United States of America
Full Time
4 weeks ago
$120,000 - $140,000 USD
H1B Sponsor Likely
Key skills
Confluent KafkaKafka cluster configurationKafka monitoringKafka topic managementMicrosoft AzureGoogle Cloud Platform.NET developmentC#API designAzure Event HubAzure Logic AppsAzure Kubernetes ServiceAzure FunctionsJavaRESTful servicesAPI securityMicroservices architectureWindows console applicationsSQLPostgreSQLCI/CD toolsAzure DevOpsGitHubTerraform scriptingInfrastructure as CodeC.NETKafkaUnit TestingAzureGCPGoogle CloudTerraformKubernetesAKSLogic AppsRESTfulAgileCI/CD
About this role
Diverse Lynx is a company seeking a Kafka Full stack Engineer to set up and manage Kafka clusters and develop cloud-based solutions. The role involves integrating Kafka with various systems, ensuring high performance and reliability, and collaborating with teams to enhance system scalability and security.
Responsibilities:
- Set up and configure Kafka clusters, components like brokers, zookeeper, control center, schema registry, and connect clusters
- Monitor cluster health and performance
- Create, delete, and configure Kafka topics
- Manage broker configurations and ensure high availability
- Diagnosing and resolve issues related to data flow, broker failures, and consumer lag
- Integrate Kafka with other systems/services (e.g., Azure/GCP services, restful APIs, databases, microservices etc)
- Provide support for Kafka-related issues in production. (Azure DevOps / GitHub). Experience working in agile development projects and sprint delivery
- Design, develop, and maintain web applications using .Net technologies across both front-end and back-end integrating with Confluent Platform
- Implement responsive user interfaces and ensure seamless integration with backend services
- Optimize application performance and ensure code quality through unit testing and code reviews
- Develop and maintain cloud-based solutions, leveraging platforms such as Azure or GCP
- Integrate and manage event-driven architectures using Confluent Kafka for scalable message streaming and processing
- Monitor and troubleshoot Confluent Kafka clusters and cloud resources to ensure high availability and reliability
- Automate deployment pipelines and manage CI/CD processes for efficient application delivery
- Implement infrastructure as code and configuration management using relevant DevOps tools
- Collaborate with development and operations teams to enhance system scalability, security, and monitoring
- Individual contributor and problem solver
Requirements:
- Hands-on experience in Confluent Kafka
- Experienced in setting up the configuration, security, and monitoring of Confluent Kafka components: brokers, zookeeper, control center, schema registry, and connect clusters
- Experienced in the development of infrastructure solutions for publisher/consumer tenants, enabling robust multi-tenant cloud services
- Experience in Microsoft Azure / Google Cloud Platform (GCP)
- Experienced in Dot Net tech stack
- Strong experience in .NET (C#) development for API design and implementation
- Proficiency with Azure cloud services, particularly Event Hub, Logic Apps (Workflows), Azure Kubernetes services (AKS), and Azure Functions (both .NET and Java implementations)
- RESTful services
- Familiarity with authentication, authorization, and security best practices for APIs
- Micro services Architecture
- Windows console apps and Services
- Proficient in SQL and PostgreSQL Database
- Hands-on experience on CI/CD tools (Azure DevOps, GitHub)
- Good understanding on terraform scripting, building pipelines and releases