Naveera Technology LLC is a global engineering partner delivering data engineering, generative AI, application development, and IT infrastructure solutions. The Data Engineering Manager will lead an AWS-to-GCP data platform migration while overseeing GCP data architecture, AI/ML and GenAI solutions, MLOps, engineering teams, stakeholder management, governance, and cloud optimization.

Responsibilities:

Design and implement AI/ML and Generative AI solutions on GCP using Vertex AI, BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Run, and related GCP-native services
Build production-grade machine learning pipelines for data preparation, model training, validation, evaluation, deployment, monitoring, retraining, and lifecycle management
Develop Generative AI and Retrieval-Augmented Generation (RAG) solutions, including enterprise search, document intelligence, AI assistants, summarization, semantic search, embeddings, vector search, and knowledge-management applications
Design scalable ingestion, transformation, chunking, embedding, indexing, and retrieval pipelines for structured and unstructured enterprise data
Implement MLOps practices using Vertex AI Pipelines, Model Registry, model endpoints, Terraform, GitHub, Cloud Build, and CI/CD pipelines
Establish standards for model versioning, experiment tracking, data and feature validation, automated testing, deployment approvals, rollback, and environment promotion
Implement monitoring for model performance, data drift, latency, reliability, inference cost, response quality, retrieval accuracy, and GenAI risks such as hallucination and prompt injection
Ensure responsible AI, data privacy, security, governance, access control, auditability, and human-review processes are incorporated into AI/ML and GenAI solutions
Partner with Data Science, Analytics, Product, BI, Security, and US-based stakeholders to identify, prioritize, and deliver high-value AI/ML and GenAI use cases
Architect and implement scalable enterprise data platforms on GCP
Design Data Lake and Lakehouse architectures using GCS and BigQuery
Define Bronze, Silver and Gold/Atomic data layers
Design scalable data ingestion, transformation and consumption frameworks
Establish standards for data modeling, partitioning, clustering and storage
Design multi-tenant and multi-location data architectures
Define schema-on-read and schema-on-write strategies
Analyze and optimize existing AWS data platforms before migration
Understand existing AWS ETL/ELT pipelines, data models, workloads and dependencies
Identify equivalent or improved GCP services for each AWS workload
Prepare technical mapping and migration plans between AWS and GCP services
Architect real-time data pipelines using Google Pub/Sub, Dataflow / Apache Beam, BigQuery, and Cloud Storage
Design high-volume event ingestion, enrichment and transformation pipelines
Implement event-driven architectures and appropriate delivery guarantees
Optimize streaming pipelines for latency, throughput and scalability
Design BigQuery streaming ingestion patterns
Implement monitoring, logging and alerting for real-time workloads
Design and implement scalable batch and real-time ETL/ELT pipelines
Migrate AWS Glue-based pipelines to appropriate GCP services
Design CDC pipelines and real-time ingestion patterns
Build orchestration workflows using Cloud Composer / Airflow
Optimize data processing jobs and query performance
Lead the end-to-end migration of enterprise data platforms from AWS to GCP
Assess existing AWS architecture, data pipelines, workloads, dependencies and operational processes
Define the target-state GCP architecture and migration roadmap
Develop migration strategies for: Amazon S3 → Google Cloud Storage; Amazon Redshift → BigQuery; AWS Glue → Dataflow / Dataproc / BigQuery; AWS Step Functions → Cloud Composer / Workflows; AWS DMS → GCP-native CDC solutions; Amazon Athena → BigQuery
Identify opportunities to modernize AWS workloads rather than performing a simple lift-and-shift migration
Define migration phases, technical dependencies, risks and rollback strategies
Lead architecture reviews and technical design discussions
Design enterprise data models for analytics and reporting
Define dimensional, normalized and denormalized data models
Develop multi-tenant data structures
Design BigQuery partitioning and clustering strategies
Optimize BigQuery SQL and query execution
Design data models supporting both real-time and batch workloads
Work closely with BI and Analytics teams to create scalable consumption models
Establish data governance and data quality standards across the GCP platform
Implement automated data quality checks and validation frameworks
Establish data lineage, metadata and ownership standards
Ensure appropriate security controls across all GCP data layers
Implement: IAM; Least-privilege access; Encryption; Service accounts; Network security; Data access policies
Work with governance and security teams to ensure compliance requirements are met
Lead infrastructure automation using Terraform
Build repeatable and secure GCP infrastructure deployments
Implement CI/CD pipelines for data engineering workloads
Automate data pipeline deployment, testing and infrastructure provisioning
Establish Dev, QA, UAT and Production deployment standards
Lead performance optimization initiatives across GCP data workloads
Optimize: BigQuery query performance; Partitioning and clustering; Dataflow pipelines; Spark workloads; Cloud Storage; Streaming workloads
Analyze AWS workloads and determine the most cost-effective GCP architecture
Develop cloud FinOps and cost optimization strategies
Establish performance benchmarks and SLAs for critical workloads
Lead and mentor a team of Data Engineers, Senior Data Engineers and Technical Leads
Provide technical direction and establish engineering standards
Conduct architecture and code reviews
Define technical roadmaps and engineering priorities
Break complex migration requirements into actionable deliverables
Track engineering progress, risks, dependencies and delivery milestones
Promote best practices around coding, testing, CI/CD, security and documentation
Mentor engineers on GCP, data architecture and modern data engineering practices
Act as the primary technical point of contact for US-based stakeholders
Work closely with Business, Product, Data Science, BI and DevOps teams
Translate business requirements into scalable technical solutions
Present architecture decisions, migration strategies and technical roadmaps
Communicate technical risks, dependencies, timelines and trade-offs
Collaborate with business teams to define operational and analytical KPIs

Requirements:

15+ years of experience in GCP Data Engineering, Data Architecture, Cloud Engineering, AI/ML Engineering, or related technology leadership roles
5+ years of strong hands-on GCP Data Engineering Experience
3+ years of strong hands-on AI/ML & Gen AI Experience
Proven experience delivering AWS-to-GCP migration projects
Strong experience designing enterprise Data Lake and Lakehouse platforms on GCP
Strong hands-on experience with BigQuery, Google Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, Dataproc, IAM, and Terraform
Experience migrating AWS data workloads, pipelines, and platforms to GCP
Strong knowledge of AWS and GCP service mapping, migration patterns, modernization strategies, and cloud architecture best practices
Experience designing, building, and deploying AI/ML solutions on GCP using Vertex AI
Hands-on experience with Generative AI, LLM-based applications, RAG architectures, embeddings, vector search, prompt engineering, and enterprise AI assistants
Strong understanding of MLOps, including model training, model registry, CI/CD/CT, model deployment, monitoring, retraining, governance, and rollback strategies
Experience implementing secure and responsible AI solutions, including data privacy, model evaluation, access controls, auditability, and governance
Expert-level SQL and strong Python and PySpark skills
Strong data modeling, data warehousing, batch processing, and real-time data engineering experience
Experience with Terraform, Git, GitHub, Cloud Build, CI/CD pipelines, and infrastructure automation
Experience managing and mentoring data engineering and cross-functional technical teams
Strong communication skills with experience working with US-based stakeholders
Python, PyTorch, TensorFlow, Scikit-learn, NLP, Deep Learning, ML Algorithms
GenAI, LLMs, GPT, Gemini, Claude, Llama, Prompt Engineering, Fine-tuning
RAG, Embeddings, Vector Databases, Semantic Search, Hybrid Search, Reranking
LangChain, LlamaIndex, LangGraph, Hugging Face, Transformers
AI Agents, Agentic Workflows, Tool/Function Calling, Multi-Agent Systems, MCP
MLflow, Kubeflow, Model Registry, Model Deployment, Monitoring, CI/CD
Vertex AI, Vertex AI Studio, Gemini, Vertex AI Pipelines, Model Garden, Vector Search
Python, FastAPI, Flask, REST APIs, SQL, Docker, Kubernetes
GCP/AWS/Azure, BigQuery, Dataflow, Spark, Databricks, Data Lakes
LLM/RAG Evaluation, Ragas, LangSmith, Guardrails, AI Governance & Security
GCP, AWS, BigQuery, GCS, AWS S3, Pub/Sub, AWS Kinesis
Advanced Python, Expert SQL, PySpark, Apache Spark, ETL, ELT, CDC
Batch & Streaming, Event-Driven Architecture, Data Pipeline Development & Optimization
Enterprise Data Lake/Lakehouse, Medallion Architecture, Data Warehousing, Data Modeling
Dimensional Modeling, Multi-Tenant Modeling, Schema-on-Read/Schema-on-Write
dbt, Apache Airflow/Cloud Composer, Dataproc, Dataflow/Apache Beam
AWS Glue, Redshift, EMR, Lambda, Kinesis, Athena, CloudWatch
BigQuery, GCS, Pub/Sub, Dataflow, Dataproc, Dataplex, Data Catalog
Terraform, Git/GitHub, Cloud Build, CI/CD, Infrastructure as Code
Data Lineage, Metadata Management, Data Quality, Monitoring, OpenLineage
Google Cloud Professional Data Engineer certification
Google Cloud Professional Machine Learning Engineer certification
Experience with Vertex AI Agent Builder, Vertex AI Search, Gemini models on Vertex AI, or enterprise Generative AI platforms
Experience with dbt, Apache Airflow, Kafka, Apache Spark, Kubernetes, Cloud Run, and API-driven architectures
Experience with Dataplex, Data Catalog, data lineage, metadata management, data governance, master data management, and data-quality frameworks
Experience supporting enterprise or regulated environments with strong data privacy, security, compliance, audit, and governance requirements

Data Engineering Manager - (GCP, AI/ML & GenAI) @ Naveera Tech, USA - Remote Work

Key skills

About this role

Responsibilities:

Requirements: