DataOps/Cloud Data Engineer - Senior
Toronto, Ontario, Canada
Full Time
20 hours ago
Key skills
PythonSQLShellPowerShellPandasETLRedshiftDatabricksAWSAzureLambdaS3GlueCloud StorageAgileScrumSDLCProject ManagementCommunicationNegotiation
About this role
We do have a job opening for
DataOps/Cloud Data Engineer - Senior
for our direct client Province of Ontario. The position details are given below:
Job Details:
Job ID: RQ11099 - DataOps/Cloud Data Engineer - Senior
Client :
Health Services Cluster
Location :
5700 Yonge St. Toronto
, Ontario
Duration: 1 year + Extension
Type: Onsite
Job Description:
Responsibilities:
-
Design, develop, test, implement, and troubleshoot: data pipelines using various tools such as Databricks, Azure Data Factory, AWS Gluecomplex data transformation procedures, data models for data warehouses and lakehouses
Experience and Skill Set Requirements:
-
Programming & scripting : Python, SQL, Linux shell, PowerShell.
-
Data manipulation/analysis using pandas and pyspark .
-
Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data
-
AWS and/or Azure services.
-
Cloud data warehouse solutions such as AWS Redshift.
-
Data lakehouse solutions such as Databricks (Delta Lake).
-
Data processing orchestration/automation
-
ETL tools (such as Azure Data Factory, AWS Glue) as well as cloud agnostic tools such as Informatica IDMC.
-
Data modeling & architecture : relational and dimensional modeling.
-
Data reporting/visualization.
-
Full SDLC from requirements gathering, design, implementation, testing to deployment and production support
-
Project management ( agile/scrum and waterfall )
-
Change and Incident management
-
Communication, presentation and negotiation skills
-
Consulting, problem-solving and decision-making skills
-
Experience working in OPS and/or public sector in general
MUST HAVES:
-
Experience using AWS Services such as Glue, StepFunctions, Lambda, S3
-
Extract/Transform/Load data using tools such as Informatica IDMC