Collaborate closely with data scientists, architects, and other stakeholders to understand and implement business requirements
Provide data engineering support for AI model development and deployment, ensuring data scientists have access to the data they need in the format they need it
Implement and optimize data transformations and ETL/ELT processes, using appropriate data engineering tools
Work with a variety of databases and data warehousing solutions to store and retrieve data efficiently
Implement monitoring, troubleshooting, and maintenance procedures for data pipelines to ensure the high quality of data and optimize performance
Participate in the creation and ongoing maintenance of documentation, including data dictionaries, data catalogues, data flow diagrams, and process documentation

Proficiency with the Python and SQL programming languages
Hands-on experience with cloud platforms like AWS, GCP, or Azure, and familiarity with big data technologies such as Hadoop or Spark
Experience working with relational databases and NoSQL databases
Strong knowledge of data structures, data modelling, and database schema design
Experience in supporting data science workloads and working with both structured and unstructured data
Familiarity with containerization technologies, such as Docker or Kubernetes
Experience with data visualization tools, such as Tableau or Power BI is a plus

Data Engineer

Key skills