Key skills
PythonShellBigQueryRedshiftSparkApache SparkAirflowdbtHadoopAWSGCPEKSEC2LambdaSNSSQSGlue
About this role
HMG AMERICA LLC is seeking an AWS Data Engineer to design and operate scalable, cloud-native data platforms on AWS. The role requires strong experience in big data distributed systems and hands-on expertise in building and optimizing data pipelines using various AWS services.
Responsibilities:
- Strong experience in big data distributed systems such as Iceberg, Hadoop, Redshift, and Hudi, and deep expertise in Apache Spark
- Hands-on experience with Shell scripting, Python, and PySpark for building and optimizing data pipelines
- Practical experience with AWS services including Glue, Lambda, Airflow, Lake Formation, EKS, EC2, SQS, and SNS
- Design and operate scalable, cloud-native data platforms on AWS
Requirements:
- strong experience in big data distributed systems such as Iceberg, Hadoop, Redshift, and Hudi
- deep expertise in Apache Spark
- hands-on experience with Shell scripting
- hands-on experience with Python
- hands-on experience with PySpark for building and optimizing data pipelines
- practical experience with AWS services including Glue, Lambda, Airflow, Lake Formation, EKS, EC2, SQS, and SNS
- design and operate scalable, cloud-native data platforms on AWS
- Experience with BigQuery
- Experience with GCP
- Experience with DBT