Sr. AWS Data Engineer

Anagh Technologies Inc is a technical consulting firm specializing in web technologies, and they are seeking a Senior AWS Data Engineer. This role involves designing, building, and maintaining high-performance data pipelines while collaborating with data architects and implementing DevOps practices.

Responsibilities:

• Design, build, and maintain high-performance ETL/ELT data pipelines using Python and PySpark
• Apply expert-level coding skills to develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets
• Take full ownership of the data workflow, including getting data from multiple sources, scrubbing, and validating data to ensure the highest quality
• Write and optimize complex, performant SQL queries for data extraction, integrity checks, and performance tuning
• Contribute to platform modernization by exploring and increasing the adoption of AI/ML, including using tools like Copilot and Claude for acceleration, and building models to fill data gaps or improve systems
• Collaborate with data architects by proposing ideas and great questions, taking ownership as the expert on data, pipelines, and systems
• Implement DevOps practices for the automated deployment and orchestration of Python applications and data pipelines (e.g., using Docker, Jenkins, Terraform)
• Hands on experience with SQL and complex performance tuning

Requirements:

• Expert-level proficiency in Python, including libraries like Pandas and NumPy
• Designing data pipelines for the data coming from multiple sources
• Solid hands-on experience with PySpark for building scalable data workflows
• Expert-level knowledge of writing complex SQL queries (Oracle or Snowflake), with proven ability to perform performance tuning on large datasets and complex database code
• Robust experience with AWS cloud services and associated data services, specifically: AWS Glue (ETL), S3, Lambda, Redshift, DynamoDB, Athena, ECS, EventBridge, OpenSearch, RDS
• Robust proficiency in ETL/ELT methodologies and tools, as well as Data Quality, Data Validation, and Anomaly Detection techniques
• Working experience with scripting and automation using Unix and Python