Deliver and stabilize our new read-only Data APIs on GCP for external partners, ensuring they meet strict rate limits and utilize cursor-based pagination.
Establish the foundational BigQuery Data Lake using Terraform, deploying automated ingestion pipelines for core platform data.
Assist in the management of the legacy Ruby on Rails auditing applications, ensuring stability while utilizing AI coding assistants to accelerate the migration to the new stack.
Play a lead role in deploying a new unified Auditing application (FastAPI/Cloud Run), merging internal and external audits into a single cloud-native architecture.
Replace manual Python Jupyter notebooks with fully automated, regulatory periodic reporting engines via Cloud Run and Workflows.
Eliminate legacy database silos by migrating transactional and analytical data into unified Cloud SQL and BigQuery schemas.
Architect and build serverless Python (FastAPI) microservices on Google Cloud Run to replace legacy monoliths.
Build and maintain robust ETL/ELT pipelines streaming data from Cloud SQL and external third-party APIs into BigQuery.
Write clean, performant SQL to transform raw data in BigQuery into curated, analysis-ready datasets (utilizing tools like Dataform or dbt).
Manage secure, high-throughput external APIs utilizing OAuth 2.0 (M2M) and API Gateway.
Utilize Terraform to deploy, secure, and monitor GCP environments.

4+ years of professional experience in Data Engineering or Backend Software Engineering.
Strong proficiency in Python and SQL.
Hands-on experience with Google Cloud Platform (GCP), specifically BigQuery, Cloud Run, and Cloud SQL.
Experience building and maintaining ETL/ELT pipelines and transforming data via modelling tools (Dataform, dbt, etc.).
Experience with Infrastructure as Code (Terraform) and CI/CD pipelines.
High proficiency in using AI coding assistants to accelerate development and infrastructure tasks.
Familiarity with FastAPI and asynchronous workers (Cloud Tasks/PubSub) (Nice to have).
Experience with modern BI tools, specifically Looker (LookML) and/or Looker Studio (Nice to have).
Experience working in regulated industries or multi-client B2B SaaS (Nice to have).
Comfortable working within legacy codebases (Ruby on Rails) while migrating them to modern architectures (Nice to have).

Mid-Senior Data Engineer, GCP, Python, BigQuery

Key skills