Data Engineer Python/C#/Kafka/Docker Distributed Systems

Job Summary

We are seeking a Data Engineer with strong software/infrastructure engineering experience to design, build, and support high-volume data pipelines and distributed systems. The ideal candidate will have hands-on experience with Python, C#, SQL, Kafka, Docker, Linux/Unix, CI/CD, and backend services.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for high-volume distributed systems.
• Build and support backend services using Python and C#/.NET.
• Develop and maintain Kafka-based data streaming and messaging solutions.
• Write, optimize, and troubleshoot complex SQL queries across relational databases and modern data platforms.
• Deploy and manage containerized applications using Docker.
• Work with Linux/Unix environments for application deployment and troubleshooting.
• Develop and maintain CI/CD pipelines using tools such as Jenkins, GitHub Actions, or similar.
• Monitor, troubleshoot, and optimize data pipelines and backend services for performance and reliability.
• Collaborate with engineering teams on scalable distributed architectures and infrastructure automation.
• Independently troubleshoot technical issues and drive solutions in a remote environment.

Required Qualifications

• 3+ years of experience in Data Engineering, Software Engineering, Infrastructure Engineering, or a related field.
• Strong hands-on experience with Python and C#/.NET.
• Strong SQL skills and experience working with large datasets.
• Experience building and supporting data pipelines.
• Hands-on experience with Apache Kafka or similar distributed messaging platforms.
• Experience with Docker and containerized applications.
• Strong experience with Linux/Unix systems.
• Experience with distributed systems, backend services, or microservices.
• Hands-on experience with CI/CD tools such as Jenkins, GitHub Actions, or similar.
• Strong troubleshooting and problem-solving skills.

Preferred Qualifications

• Experience with Java.
• Experience with Snowflake, BigQuery, or other cloud data platforms.
• Experience with AWS, Azure, or Google Cloud Platform.
• Experience with Kubernetes.
• Experience with Terraform or Infrastructure as Code.
• Experience with Airflow, Cloud Composer, or similar workflow orchestration tools.
• Experience with Apache Beam or similar data processing frameworks.
• Experience with cloud object/blob storage.

Candidates with strong backend/software engineering experience combined with data engineering are highly encouraged to apply.