Creation and maintenance of data pipelines between layers in the Data Lake, including optimization;
Preparation and modeling of databases;
Ingestion of databases into the Data Lake;
Ensure the data quality of developed databases according to analytics standards/policies;
Data analysis and requirements gathering from the business;
Create and maintain data pipelines between layers in the Data Lake;
Develop and model databases for dashboards, reports, algorithms and/or refined datasets;
Document daily activity updates in JIRA;
Document metadata and business rules using the standard internal template;
Understand and gather requirements for databases/variables with the business;
Perform ingestion of databases into the Data Lake;
Perform data quality analysis and ensure analytics standards are being applied;
Version developed code.

Minimum of 4 years of corporate experience with GCP;
Experience with data architecture projects in GCP (BI and Big Data);
Experience with data integration models (APIs) and complex data integration environments focused on GCP (BI and Big Data);
Experience with Agile methodology and JIRA;
Strong knowledge of languages and technologies such as Python, Spark, Streaming, SQL, NoSQL and Git;
Strong knowledge of data modeling and Data Lake concepts;
Strong knowledge of version control tools such as GitHub and Bitbucket;
Strong experience with GCP services including:
Cloud Storage;
Composer / Airflow;
Dataflow;
Dataproc;
BigQuery;
Cloud Functions;
Cloud Run.

Data Engineer, GCP, Mid-level

Key skills