Fractal is a strategic AI partner to Fortune 500 companies, aiming to enhance human decision-making in enterprises. They are seeking a Senior Data Engineer - AWS to design and build scalable data pipelines on AWS, guide client teams through data platform best practices, and act as a technical consultant to solve complex data challenges.
Responsibilities:
- Design, build, and maintain scalable data pipelines using AWS services such as S3, Lambda, Step Functions, Glue (Serverless), and EventBridge, landing curated data in Snowflake
- Develop and optimize ETL/ELT workflows using PySpark, Parquet file formats, and Apache Iceberg table formats
- Build and maintain semantic layers in Snowflake to enable consistent, governed, business-ready analytics
- Manage metadata and data discovery using AWS Glue Catalog and Glue Crawlers, and enable analytics through Athena
- Implement event-driven architectures with SNS and EventBridge for data processing and notifications
- Enable secure data lake operations using AWS Lake Formation for access control and governance
- Support data ingestion and migration initiatives, including migrating data from Teradata to S3
- Ensure reliable deployments and infrastructure automation using uDeploy, CloudFormation, Terraform, CDK, and GitHub
- Collaborate on cost reporting, monitoring, and optimization, and research emerging AWS features (e.g., S3 Tables) to improve performance, scalability, and cost efficiency
- Act as a hands-on subject-matter expert and technical consultant for client stakeholders and their engineering teams
- Mentor and upskill client engineers, sharing best practices for AWS and Snowflake data engineering and raising the technical bar across the team
- Partner with cross-functional teams to translate business problems into robust, scalable, end-to-end data solutions
Requirements:
- Expert-level data engineer with extensive, hands-on experience designing and building production data platforms on AWS
- Proven ability to build end-to-end data pipelines from the ground up—ingestion, transformation, and delivery—not just maintain existing ones
- Hands-on experience designing and building semantic layers in Snowflake
- Track record of acting as a technical subject-matter expert or consultant, advising and upskilling client teams and engineering stakeholders
- Comfortable being the most senior technical voice in the room and translating complex engineering concepts for both technical and business audiences
- Excellent problem-solving skills and the ability to collaborate effectively across cross-functional teams
- Self-directed and proactive, able to balance multiple workstreams with a focus on quality and delivery
- Deep, hands-on expertise with core AWS data services: S3, Lambda, Step Functions, Glue (Serverless), Glue Catalog, Glue Crawlers, Athena, SNS, EventBridge, and Lake Formation
- Strong proficiency in PySpark and building production ETL/ELT workflows, including with columnar and open table formats such as Parquet and Apache Iceberg
- Hands-on Snowflake experience, including designing and building semantic layers
- Practical knowledge of infrastructure-as-code and CI/CD tooling: CloudFormation, Terraform, CDK, uDeploy, and GitHub
- Experience with large-scale data migrations (e.g., migrating from Teradata to S3)
- Strong understanding of cost management, monitoring, and reporting in AWS environments
- Working knowledge of data lake architectures, metadata management, and data governance
- Bachelor's degree with a quantitative or technical emphasis: Computer Science, Engineering, Data Science, Mathematics, Information Systems, or a related field
- Extensive hands-on data engineering experience, typically 7+ years, with deep expertise in AWS and Snowflake
- Master's degree desirable