Key skills
ApacheAzurePythonSparkAnalyticsDatabricksApache SparkAzure DevOpsAzure MonitorCI/CD
About this role
Role Overview
- Design, develop and maintain reliable, dynamic, automated data pipelines
- Collect, integrate and transform structured and unstructured data (JSON, CSV, Parquet)
- Monitor, optimize and make available data pipelines
- Manage data quality, governance and traceability (DataOps)
- Work closely with data scientists
- Help analyze and resolve data-related issues
- Use and maintain the CI/CD pipelines deployed through Azure DevOps
Requirements
- Bachelor's degree in a related field
- A minimum of four years of relevant experience
- Experience elaborating data solutions
- Knowledge of French is required for positions in Quebec
- Proficiency in Azure Data Factory and Azure Synapse Analytics and knowledge of Azure Databricks
- Proficiency in Azure Purview, Azure Monitor and Log Analytics
- Knowledge of Python
- Knowledge of data processing and storage technologies, including Apache Spark and Delta Lake
Tech Stack
- Apache
- Azure
- Python
- Spark
Benefits
- Competitive salary and annual bonus
- 4 weeks of flexible vacation starting in the first year
- Defined benefit pension plan that provides predictable, stable income throughout retirement
- Group insurance including telemedicine
- Reimbursement of health and wellness expenses and telework equipment