Big Data Engineer – Real-Time Streaming
United States
Full-time
9 hours ago
$74,000 - $111,000 USD
No H1B
Key skills
PythonSQLETLData EngineeringData LakeAnalyticsTableauSnowflakeKafkaApache KafkaAWSTerraformEKSCloudFormationS3IAMAthenaCI/CD
About this role
Mogi I/O is a company focused on OTT, podcast, and short video applications, and they are seeking an experienced Big Data Engineer to design and optimize scalable cloud-based data platforms on AWS. The role involves building enterprise-grade data lakes and collaborating with teams to deliver high-performance data ecosystems.
Responsibilities:
- Design, develop, and maintain scalable data pipelines using AWS services
- Build and manage cloud-based data lakes using Amazon S3
- Implement and optimize Snowflake data models for analytics and reporting
- Develop real-time streaming solutions using Apache Kafka
- Integrate enterprise systems using MuleSoft APIs and connectors
- Deploy and manage containerized data workloads on Amazon EKS
- Write and optimize complex queries using Amazon Athena
- Monitor system performance and ensure reliability and scalability
- Collaborate with analytics and business teams to support reporting needs
- Implement best practices for security, governance, and data quality
- Support CI/CD processes for data engineering workflows
- Stay updated with emerging cloud and data engineering technologies
Requirements:
- 12+ years of experience in Data Engineering or related field
- Strong expertise in AWS Cloud Services (S3, Athena, EKS, IAM, Monitoring)
- Hands-on experience with Snowflake (data modeling, performance tuning, reporting layer)
- Experience with Apache Kafka (real-time streaming, producers/consumers, pipelines)
- MuleSoft experience (API-based integrations, enterprise data flows)
- Experience managing containerized deployments on Amazon EKS
- Strong SQL skills and experience with Amazon Athena
- Experience building S3-based Data Lake architectures
- Reporting and analytics experience using Snowflake and Tableau
- Strong analytical and problem-solving skills
- Python-based ETL development experience
- CI/CD pipeline implementation for data workflows
- Infrastructure as Code (Terraform or CloudFormation)
- Experience working with large-scale, high-volume data systems