
Job Description:
We are seeking an experienced Big Data Developer with strong expertise in Apache Spark, Hadoop, AWS, Python/Scala, and SQL to build and optimize large-scale data processing solutions.
Required Skills:
8+ years of Big Data/Data Engineering experience
Strong hands-on experience with Apache Spark
Hadoop Ecosystem: Hadoop, Hive, Trino
Strong programming skills in Python or Scala
Advanced SQL (Joins, Window Functions, Aggregations, Query Optimization)
Experience building and supporting large-scale ETL/Data Pipelines
AWS Services: S3, EMR, Glue, Athena, Lambda
Experience troubleshooting Spark jobs, data skew, performance tuning, and large-volume data processing
Knowledge of Agile/Scrum development methodologies
Experience with AI-assisted development tools (GitHub Copilot, ChatGPT, Claude, Amazon Q)
Preferred Skills:
EKS / Kubernetes
CI/CD Pipelines
Test Automation
Production Data Pipeline Support
AWS Certifications
Financial Services Experience