To design, development, and enhancement of our AWS data environment.
Requirements
Experience setting up AWS Data Platform – AWS CloudFormation, DevelopmentEndPoints, AWS Glue, EMR and Jupyter/Sagemaker Notebooks, Redshift, S3, and EC2 instances.
Track record of successfully building scalable Data Lake solutions that connects to distributed data storage using multiple data connectors.
Must have a background in data engineering – Data Warehouse Development experience would be perfect
Must have strong skills in SQL, Python, PySpark, and AWS
Experience in designing, developing, optimizing and troubleshooting complex Data Pipelines using Spark cluster
Ability to lead proofs-of-concepts and then effectively transition and scale those concepts into production at scale through, engineering, deployment and commercialization.
Serve as an expert; envision and integrate emerging data technologies, anticipate new trends to solve complex business and technical problems.