Senior Data Engineer

K&K Global Talent Solutions Inc. is an international recruiting agency that has been providing technical resources in the Canada and the USA region since 1993. They are seeking a Senior Data Engineer to design and implement scalable data architectures, manage databases, and collaborate with stakeholders to optimize data processes.

Responsibilities:

• Associate should be able to create Azure Policies and baseline rules
• Also able to handle KPIs and corresponding dashboard development and maintenance
• Should have excellent communication to handle the Office Hours or the support work
• Design and implement scalable data architectures, ensuring high availability, reliability, and performance of data systems
• Strategically architect, design, develop, and implement efficient information systems and/or operations systems in support of core enterprise functions
• Understand business requirements, study existing application landscape, and identify redundant/ineffective systems
• Build, maintain, and optimize data pipelines for extracting, transforming, and loading (ETL/ELT) data from various sources into data warehouses or data lakes
• Create data migration/system integration strategies
• Conceptualize technical solutions to complex problems and maximize benefit of systems investments
• Manage and optimize databases (both SQL and NoSQL), ensuring data consistency, quality, and security
• Integrate data from disparate sources, including APIs, flat files, and third-party data, ensuring comprehensive data availability for analytics and reporting
• Implement and monitor data quality measures to ensure accuracy, completeness, and reliability of data
• Design and develop/maintain the reports
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs
• Work closely with clients to gain organizational commitment for all systems and software plans, as well as evaluate and guide the selection of technologies required to complete those plans
• Automate repetitive tasks to improve data processing efficiency and reduce manual intervention
• Monitor system performance to detect and resolve problems during deployment and support change management
• Maintain comprehensive documentation for data processes, systems, and pipelines to facilitate knowledge sharing and onboarding
• Design and develop the reports mostly in Power BI

Requirements:

• Microsoft Azure
• Kusto
• PL/SQL
• Microsoft Power BI
• Microsoft SQL Server 2012
• Business Intelligence
• Create Azure Policies and baseline rules
• Handle KPIs and corresponding dashboard development and maintenance
• Excellent communication skills to handle Office Hours or support work
• Design and implement scalable data architectures
• Ensure high availability, reliability, and performance of data systems
• Strategically architect, design, develop, and implement efficient information systems and/or operations systems
• Understand business requirements and study existing application landscape
• Identify redundant/ineffective systems
• Build, maintain, and optimize data pipelines for ETL/ELT
• Create data migration/system integration strategies
• Conceptualize technical solutions to complex problems
• Maximize benefit of systems investments
• Manage and optimize databases (both SQL and NoSQL)
• Ensure data consistency, quality, and security
• Integrate data from disparate sources, including APIs, flat files, and third-party data
• Implement and monitor data quality measures
• Design and develop/maintain reports
• Work closely with data scientists, analysts, and other stakeholders
• Deliver solutions that meet business needs
• Work closely with clients to gain organizational commitment for systems and software plans
• Evaluate and guide the selection of technologies required to complete plans
• Automate repetitive tasks to improve data processing efficiency
• Monitor system performance to detect and resolve problems
• Maintain comprehensive documentation for data processes, systems, and pipelines
• Design and develop reports mostly in Power BI