Data Platform Engineering Lead

Role Overview

• Lead and mentor a team of data architects and data engineers, fostering a collaborative and high-performance culture.
• Oversee the design, development, and maintenance of scalable and robust data architectures and pipelines, ensuring efficient data processing, storage, and retrieval, including data ingestion, storage, processing, management, and analytics components.
• Develop and maintain the architecture of the data platform, ensuring it meets the needs of various business units and supports future growth.
• Define the technology stack and architectural standards for the data platform, ensuring alignment with industry best practices and emerging trends.
• Develop strategies for integrating diverse data sources, ensuring seamless data flow and high data quality.
• Design solutions that can scale with growing data volumes and ensure optimal performance across the data platform.
• Continuously monitor and optimize platform performance to handle large-scale data efficiently.
• Implement robust data security measures and ensure compliance with relevant regulations and industry standards.
• Work closely with data scientists, analysts, product managers, and other stakeholders to understand data requirements and deliver effective solutions.
• Stay up-to-date with the latest advancements in data technologies and drive continuous innovation within the data platform.
• Maintain comprehensive documentation of data pipelines, ETL processes, and architectural decisions.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
• 7+ years of experience in data engineering or a similar role, with at least 3 years in a leadership position.
• Strong experience with data platform reference architectures (e.g., Lambda architecture, Data Mesh).
• Deep knowledge of big data technologies (e.g., Hadoop, Spark, Kafka) and data warehousing solutions (e.g., Redshift, Snowflake).
• Extensive experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services, with a focus on Google Cloud.
• Google Cloud certification is preferred.
• Good knowledge of relevant security frameworks and standards.
• Proficiency in programming languages such as Python, Java, or Scala.
• Strong understanding of database management systems (e.g., SQL, NoSQL, NewSQL).
• Experience with SQL and relational database systems (e.g., MySQL, PostgreSQL, SQL Server).
• Knowledge of data integration tools and frameworks (e.g., Apache NiFi, Talend, Informatica).
• Familiarity with data modeling, data warehousing, and data governance practices.
• Experience with Infrastructure-as-Code (IaC) tools (e.g., Ansible, Terraform), data pipeline orchestration (e.g., Airflow), dashboard/log exploration tools (e.g., Streamlit, Dash), data extraction and APIs (e.g., PostGIS, Kafka, Airflow, FastAPI), pandas, scikit-learn, and Docker.
• Solid knowledge of DevOps best practices and tools: Git, CI/CD, telemetry, and monitoring.
• Strong analytical and problem-solving skills with a focus on delivering scalable and efficient data solutions.
• Proven leadership skills with experience in building and leading high-performing teams.
• Excellent verbal and written communication skills, with the ability to effectively collaborate with technical and non-technical stakeholders.
• Strong project management skills with the ability to manage multiple projects and priorities simultaneously.
• High attention to detail and a commitment to ensuring data quality and accuracy.
• Ability to work in a fast-paced, dynamic environment and manage multiple priorities simultaneously.

Tech Stack

• Airflow
• Amazon Redshift
• Ansible
• Apache
• AWS
• Azure
• Cloud
• Docker
• ETL
• Hadoop
• Informatica
• Java
• Kafka
• MySQL
• NoSQL
• Pandas
• PostGIS
• Postgres
• Python
• Scala
• Scikit-Learn
• Spark
• SQL
• Terraform

Benefits

• An excellent work environment and an opportunity to create a real impact in the world.
• A truly high-tech, state-of-the-art engineering company with a flat structure and no politics.
• Work with the very latest technologies in Data & AI, including Edge AI and Swarming—both within our software platforms and within our embedded on-board systems.
• Flexible work arrangements.
• Professional development opportunities.
• Collaborative and inclusive work environment.
• Salary commensurate with proven experience.