Lead Data Engineer

ApTask is a leading global provider of workforce solutions and talent acquisition services, dedicated to shaping the future of work. They are seeking a Lead Data Engineer with expertise in Google Cloud Platform and BigQuery to drive data processing and transformation initiatives.

Responsibilities:

• Python: Programming, ETL scripting, business logic implementation, and experience with data manipulation libraries (pandas, NumPy)
• PL/SQL: Data transformation, view/table creation, and persistence layer interactions
• Data Transformation Tools: DBT (Data Build Tool) for creating models and documentation
• Orchestration: Airflow, Dataflow, and Cloud Composer
• Data Ingestion: Cloud Functions, Cloud Run, SFTP mechanisms, and Falcon (for database connections)
• Google Cloud Platform (GCP) and BigQuery experience
• Experience with cloud-native data processing
• Understanding of householding and data segmentation logic
• Ability to work with streaming and batch data
• Experience with data integration from multiple sources (e.g., Reltio, external databases)
• Familiarity with incremental data loading and partitioning strategies

Requirements:

• Python: Programming, ETL scripting, business logic implementation, and experience with data manipulation libraries (pandas, NumPy)
• PL/SQL: Data transformation, view/table creation, and persistence layer interactions
• Data Transformation Tools: DBT (Data Build Tool) for creating models and documentation
• Orchestration: Airflow, Dataflow, and Cloud Composer
• Data Ingestion: Cloud Functions, Cloud Run, SFTP mechanisms, and Falcon (for database connections)
• Google Cloud Platform (GCP) and BigQuery experience
• Experience with cloud-native data processing
• Understanding of householding and data segmentation logic
• Ability to work with streaming and batch data
• Experience with data integration from multiple sources (e.g., Reltio, external databases)
• Familiarity with incremental data loading and partitioning strategies