Data Pipeline Manager

Role Overview

• As the Data Pipeline Manager, you will lead a team responsible for building and maintaining scalable, reliable data pipelines that support enterprise reporting and analytics.
• You will oversee the development and optimisation of Talend-based data workflows, ensuring high data quality and availability across systems such as Qlik and our central databases.
• Collaborating with data stewards, analysts, and support teams, you will guide the implementation of best practices in data integration, transformation, and validation.
• This role is critical in ensuring the stability and performance of data operations that underpin business insights and application functionality.
• Liaise with VDI and IT to allow the smooth transition of data through Qlik Talend
• Put a structure in place to create an environment for Qlik Talend and GitHub to allow easy development of pipelines.
• Ensure that Qlik Talend Developers are allocated suitable well-defined requirements.
• Act as a point of escalation for any Qlik Talend questions.
• Make sure that the Qlik environment is running and that automated pipelines can be ran.
• Generate and consistently improve Qlik Talend pipelines.
• Create and maintain pipelines for each customer’s data.
• Create a reusable toolkit of code for validation and transformations that can be used by other customers.
• Generate automated validation checks on the received data so that it can be handled more efficiently before loading into the database.
• Become an expert in using the Qlik Talend Studio software so that any data transformations requested by the business can be performed.
• Use Java code to perform advanced manipulation of the data.
• Integrate Api’s into the Talend Studio.
• Keep up to date with improvements in the Qlik Talend Studio to improve data handling.
• Ensure standards are maintained.
• Make sure that data is handled correctly by Qlik Talend and that poor data is not loaded into the Database.
• Use GitHub to allow source control of the created pipelines.
• Implement Qlik Talend with Data Streams
• Liaise with Account Managers to make sure that pipelines can process the data received from Customers.
• Liaise with business to understand customer requirements, and to keep the business informed of pipeline progress.
• Provide feedback to users about the quality of the customers’ data.
• Establish and maintain a data stewardship model that works for the operational team.

Requirements

• Management experience
• Excellent written and verbal communication skills
• Great attention to detail
• Excellent organisational skills.
• Strong analytical and problem-solving abilities
• Experience using Microsoft packages, including Excel
• Knowledge of ETL tools and data pipelines, SQL, Java and GitHub is required
• Applicants should be educated to degree level or possess solid work experience in related field.

Tech Stack

• ETL
• Java
• SQL

Benefits

• Dual Location / Hybrid worker Status
• Participation in our annual Incentive Plan (VIP)
• 25 days annual leave plus bank holidays, volunteering day and your birthday off!
• Option to buy and sell up to 9 days annual leave
• Access to voluntary benefits including private medical insurance, cycle to work scheme, and subsidised gym membership
• Automatic inclusion in Life Assurance, Critical Illness and Disability Income protection schemes
• Pension scheme up to 8% employer contribution
• Access to reward & discount platform
• Wellbeing initiatives