Senior Data Engineer

Role Overview

• Design and implement scalable, efficient data pipelines and infrastructure to support business intelligence and analytics initiatives (Databricks, Power Automate).
• Lead data engineering to build robust, high-performance data systems that align with business objectives as well as enabling fit for GenAI and Machine Learning datasets.
• Collaborate with stakeholders across the organization to understand data requirements and ensure data solutions meet business needs and serve appropriate data.
• Oversee the development, optimization, and maintenance of data pipelines, ensuring data is collected, processed, and made available for analysis in a timely and accurate manner.
• Ensure data quality, integrity, and governance across all data systems by implementing best practices for data validation, security, and privacy (Unity Catalog in Databricks).
• Develop and maintain ETL processes to integrate data from various sources into centralized data warehouses and data lakes.
• Partner with analysts and business teams to design data architectures that enable effective reporting, analysis, and decision-making.
• Act as the primary point of contact for data engineering, ensuring smooth communication between technical teams and business stakeholders.
• Translate business needs into technical specifications, ensuring the data infrastructure supports both current and future analytics requirements.
• Monitor and optimize the performance of data systems and pipelines, ensuring they meet service level agreements (SLAs) and business expectations.
• Continuously evaluate and implement new technologies and tools to enhance data processing capabilities and improve overall system performance (Zapier, Genie AI).
• Continuously identify areas for process improvements and implement automation to enhance the efficiency and scalability of data workflows.
• Ensure data storage and processing solutions are optimized for cost and performance, adapting to evolving business needs.

Requirements

• 5+ years of experience in data engineering, data warehousing, or a similar role.
• Proven experience in designing, building, and maintaining large-scale data systems and workflows in cloud environments.
• Expertise in SQL and Python, or other programming languages used for data processing and pipeline development.
• Strong knowledge of data warehousing solutions (e.g., Snowflake, Databricks, Redshift, BigQuery) and cloud platforms (e.g. AWS, GCP, Azure).
• Experience with ETL tools, data integration platforms, and data pipeline orchestration tools (e.g. Apache Airflow, Talend, dbt).
• Knowledge of reporting and Business Intelligence Tools (e.g. Power BI, Tableau, Cognos)
• Familiarity with data governance principles, data security, and privacy standards.

Tech Stack

• Airflow
• Amazon Redshift
• Apache
• AWS
• Azure
• BigQuery
• Cloud
• Cognos
• ETL
• Google Cloud Platform
• Python
• SQL
• Tableau
• Unity