Data engineer

Job Summary

We are seeking a highly skilled Data Engineer to design, develop, and optimize scalable data pipelines that process large volumes of data. The ideal candidate will have deep expertise in Python, PySpark, AWS Glue, and Amazon EMR, along with experience implementing data quality and lineage frameworks to ensure data integrity across enterprise data platforms.

Required Qualifications

• 5+ years of Data Engineering experience.
• Expert-level proficiency in Python and PySpark.
• Strong hands-on experience with AWS Glue and Amazon EMR.
• Experience building and supporting large-scale data pipelines (terabyte-scale preferred).
• Strong knowledge of Amazon S3 Data Lakes.
• Experience with Data Quality and Data Lineage implementations.
• Strong experience with PostgreSQL, MySQL, or AWS RDS.
• Excellent problem-solving and performance-tuning skills.

Preferred Qualifications

• Experience in the Financial Services industry.
• Hands-on experience with Kafka or AWS Kinesis for real-time data processing.
• Knowledge of CI/CD pipelines for data engineering workflows.
• Experience with Infrastructure as Code (IaC) tools such as Terraform or CloudFormation.
• Familiarity with Agile development methodologies.