Data Engineer

Manulife Financial Corporation is a leading international financial services provider, helping people make their decisions easier and lives better. They are seeking a Data Engineer to design and maintain data pipelines, migrate data architectures to cloud platforms, and collaborate with data scientists to enhance data-driven decision-making.

Responsibilities:

• Discover hidden information using massive datasets and leveraging advanced data analytics and machine learning techniques to enable data-driven strategic decision-making
• Design, build and maintain data pipelines for ETL/ELT processes for large scale datasets
• Migrate legacy data architectures to Azure and AWS leveraging cloud-native services for scalability
• Develop and optimize Spark based pipelines using databricks for large-scale data processing and Synpse for creating data pipelines
• Automate complex workflows for data ingestion, transformation, and delivery using Airflow
• Create interactive dashboards to communicate insights to business stakeholders
• Design and manage relational MSSQL, PostgreSQL, and NoSQL (MongoDB) databases for diverse use cases
• Implement secure and compliant pipelines including governance policies for sensitive data
• Integrate various data sources to streamline data processing and analysis using REST APIs
• Collaborate with data scientists to build pipelines for training and deploying machine learning models
• Work with cross-functional teams to gather requirements and deliver tailored data solutions
• Set up monitoring and alerting systems to ensure data pipeline health and reliability
• Use Docker to built and deploy applications and ensure consistency across development and production environments
• Identify inefficiencies in existing workflows and implement process improvements to reduce costs

Requirements:

• Master's degree or foreign equivalent in Computer Science, Electrical Engineering, Electronic Engineering, Information Systems or related field and 3 years of experience with Python, SQL, and ETL processes
• 3 years of experience with cloud platforms including AWS and Azure
• 3 years of experience using distributed computing frameworks including Spark, Databricks and Synapse
• 2 years of experience with orchestration tools including Apache Airflow
• 3 years of experience with data visualization tools including Power BI and Tableau
• 3 years of experience with database systems including SQL and NoSQL
• 3 years of experience using data governance, compliance, and security practices
• 3 years of experience with REST APIs and integration frameworks
• 3 years of machine learning and predictive modeling experience
• 3 years of experience with monitoring tools including Splunk and CloudWatch
• 2 years of experience using DevOps practices and containerization