Key skills
AWS Cloud PlatformPythonSQLSparkAWS EC2AWS S3AWS EMRAWS KinesisAWS RDSAWS RedshiftAWS SpectrumAWS LambdaAWS GlueAWS AthenaAWS API GatewayData PipelinesData Extraction, Transformation, and Loading (ETL)Database Platforms - RedshiftDatabase Platforms - Amazon RDSDatabase Platforms - CassandraDatabase Platforms - SnowflakeDatabase Platforms - PostgreSQLDatabase Platforms - DatabricksMachine LearningGenAIData EngineeringAnalyticsSnowflakeRedshiftDatabricksCassandraAWSEC2LambdaS3RDSGlueAthenaAPI GatewayKinesisPostgreSQL
About this role
Protagona is seeking a Data Engineer to join their talented team responsible for deploying and configuring cloud resources in AWS. The role involves evaluating business needs, translating requirements into technical specifications, and developing data pipelines to support client engagements.
Responsibilities:
- Work with the team to evaluate business needs and priorities, liaise with key business partners and address team needs related to data systems and management
- Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components and enhancements
- Participate in project planning; identifying milestones, deliverables and resource requirements; tracks activities and task execution
- Generate design, development, test plans, detailed functional specifications documents, user interface design, and process flow charts for execution of programming
- Develop data pipelines / APIs using a variety of tools, including but not limited to Python, SQL, Spark and various AWS services
- Use an analytical, data-driven approach to drive a deep understanding of fast changing business
- Build large-scale batch and real-time data pipelines with data processing frameworks in AWS
- Moving data from on-prem to cloud and cloud data conversions
Requirements:
- Experience in data engineering with an emphasis on data pipelines, analytics and reporting
- Exposure to the AWS Cloud Platform
- Experience in SQL, data transformations, and troubleshooting across at least one database Platform (Redshift, Amazon RDS, Cassandra, Snowflake, PostgreSQL, Databricks, etc.)
- Experience in the design and build of data extraction, transformation, and loading processes by writing custom data pipelines
- Experience in a scripting languages such as Python
- Experience designing and building solutions utilizing various Cloud services such as EC2, S3, EMR, Kinesis, RDS, Redshift/Spectrum, Lambda, Glue, Athena, API gateway, etc
- AWS Certification - Solutions Architect Pro or Data Specialty
- Machine Learning experience
- Advanced GenAI experience