Ensure the reliable and timely execution of daily data pipelines and scheduled workflows.
Operate and maintain internal data services, including ingestion layers, OLAP/lake storage, materialised views, and task dependencies.
Contribute to CI/CD workflows for data pipelines and participate in deployments, version management, and change control.
Monitor orchestration systems (e.g., Airflow), troubleshoot pipeline failures, delays, and anomalies, and drive continuous performance improvements.
Implement and maintain data quality checks, anomaly detection, schema validation, and audit processes.
Collaborate with Data Engineers on table lifecycle management, storage optimisation, partitioning strategies, and schema evolution.
Work with IT Infrastructure and IT Operations teams to improve platform observability, including logging, metrics, and alerting.
Develop and maintain SOPs, platform standards, best practices, and troubleshooting documentation.
Provide operational support to internal users (DE/DA/DS/Ops) for issues such as query performance, missing data, or inconsistent KPIs.
Requirements
2+ years of experience in Data Ops, Data Engineering, BI Engineering, or a similar operational data role.
Experience with CI/CD workflows, Docker, Kubernetes, or other DevOps-related practices.
Hands-on experience with workflow orchestration tools such as Airflow (or equivalent).
Familiarity with mainstream data engineering technologies such as Kafka, Spark, Flink, Delta Lake, Iceberg, Hudi, ClickHouse, or Doris.
Good understanding of data warehousing concepts, including partitioning, schema evolution, table lifecycle management, and OLAP vs. data lake architectures.
Strong SQL skills and familiarity with Python for scripting, automation, or validation.
Strong debugging and problem-solving skills, especially for data anomalies and pipeline failures.
Comfortable working cross-functionally with DE/Infra/Ops/DA/DS teams in a fast-paced environment.