AWS Data platform Engineer

American Unit, Inc is seeking a highly skilled AWS Data Platform Engineer to design, build, and optimize modern, scalable, serverless data platforms. This role requires deep expertise in AWS Data Lake architecture, Glue, Lake Formation, PySpark, and Infrastructure-as-Code, with responsibilities including developing serverless architectures and monitoring data pipelines.

Responsibilities:

• Design and manage AWS Data Lakes using:
• Amazon S3 (raw, curated, processed zones)
• AWS Glue Jobs, Crawlers, Data Catalog
• AWS Lake Formation governance
• Implement:
• Row/column-level access control
• LF-TBAC (Tag-Based Access Control)
• Cross-account data sharing
• Optimize partitioning, schema evolution, and metadata strategies
• Build batch and near-real-time ingestion pipelines
• Develop serverless architectures using:
• Lambda
• API Gateway
• EventBridge
• SNS/SQS
• Step Functions
• Build and optimize Glue ETL jobs using Python & PySpark
• Implement scalable, cost-efficient event-driven architectures
• Configure CloudWatch Logs, Metrics & Dashboards
• Implement alarms for:
• Lambda failures
• Glue job errors
• SLA breaches
• Use CloudWatch Log Insights for analytics
• Configure AWS X-Ray for tracing
• Develop ETL pipelines using PySpark (Glue/EMR)
• Optimize Spark workloads for cost and performance
• Troubleshoot distributed processing issues
• Infrastructure-as-Code using:
• Terraform
• CloudFormation / CDK (optional)
• Secure IAM roles and policies
• Automate provisioning and CI/CD deployments

Requirements:

• 6+ years of Data Engineering experience
• Strong AWS Data Lake implementation experience
• Hands-on with AWS Glue
• Hands-on with Lake Formation
• Hands-on with PySpark
• Hands-on with Lambda
• Experience with Terraform (IaC)
• Strong Python development skills
• Experience implementing monitoring & logging pipelines
• Understanding of data governance & security
• 8+ Years Preferred
• CloudFormation / CDK (optional)