Engineering Manager (AWS to GCP Data Migration, AI/ML & GenAI) @ Naveera Tech, USA - Remote Work

United States of America

Full Time

4 weeks ago

Visa Sponsor

Key skills

AWS Data EngineeringGoogle Cloud Platform (GCP) Data EngineeringAWS-to-GCP Cloud MigrationEnterprise Data Lakehouse ArchitectureBigQueryPythonSQLPySpark/Apache SparkETL/ELT and Batch/Streaming Data ProcessingGenerative AI and RAG ArchitecturesMLOps with Vertex AITerraform and CI/CDEngineering LeadershipStakeholder ManagementAIMachine LearningMLGenerative AIGenAILLMGeminiRAGMLOpsETLELTData EngineeringData WarehousingData LakeAnalyticsBIRedshiftSparkApache SparkKafkaAirflowApache AirflowdbtAWSGCPGoogle CloudTerraformKubernetesServerlessS3IAMGlueAthenaCloud RunPub/SubCloud StorageGCSCloud BuildVertex AIcomposerGitGitHubPerformance OptimizationCI/CDLeadershipMentoringCommunicationNetwork Security

About this role

Naveera Technology LLC is an engineering partner delivering data engineering, generative AI, application development, and IT infrastructure solutions. The Engineering Manager will lead a large-scale AWS-to-GCP data platform migration, architect enterprise data and AI/ML platforms, manage engineering teams, and coordinate with stakeholders to deliver secure, scalable, and cost-effective solutions.

Responsibilities:

Lead the end-to-end migration of enterprise data platforms from AWS to GCP
Assess existing AWS architecture, data pipelines, workloads, dependencies and operational processes
Define the target-state GCP architecture and migration roadmap
Develop migration strategies for Amazon S3 → Google Cloud Storage, Amazon Redshift → BigQuery, AWS Glue → Dataflow / Dataproc / BigQuery, AWS Step Functions → Cloud Composer / Workflows, AWS DMS → GCP-native CDC solutions, and Amazon Athena → BigQuery
Identify opportunities to modernize AWS workloads rather than performing a simple lift-and-shift migration
Define migration phases, technical dependencies, risks and rollback strategies
Lead architecture reviews and technical design discussions
Architect and implement scalable enterprise data platforms on GCP
Design Data Lake and Lakehouse architectures using GCS and BigQuery
Define Bronze, Silver and Gold/Atomic data layers
Design scalable data ingestion, transformation and consumption frameworks
Establish standards for data modeling, partitioning, clustering and storage
Design multi-tenant and multi-location data architectures
Define schema-on-read and schema-on-write strategies
Analyze and optimize existing AWS data platforms before migration
Understand existing AWS ETL/ELT pipelines, data models, workloads and dependencies
Identify equivalent or improved GCP services for each AWS workload
Prepare technical mapping and migration plans between AWS and GCP services
Architect real-time data pipelines using Google Pub/Sub, Dataflow / Apache Beam, BigQuery, and Cloud Storage
Design high-volume event ingestion, enrichment and transformation pipelines
Implement event-driven architectures and appropriate delivery guarantees
Optimize streaming pipelines for latency, throughput and scalability
Design BigQuery streaming ingestion patterns
Implement monitoring, logging and alerting for real-time workloads
Design and implement scalable batch and real-time ETL/ELT pipelines
Migrate AWS Glue-based pipelines to appropriate GCP services
Develop transformation frameworks using Python, PySpark, SQL, Dataflow / Apache Beam, BigQuery, and dbt
Design CDC pipelines and real-time ingestion patterns
Build orchestration workflows using Cloud Composer / Airflow
Optimize data processing jobs and query performance
Design enterprise data models for analytics and reporting
Define dimensional, normalized and denormalized data models
Develop multi-tenant data structures
Design BigQuery partitioning and clustering strategies
Optimize BigQuery SQL and query execution
Design data models supporting both real-time and batch workloads
Work closely with BI and Analytics teams to create scalable consumption models
Establish data governance and data quality standards across the GCP platform
Implement automated data quality checks and validation frameworks
Establish data lineage, metadata and ownership standards
Ensure appropriate security controls across all GCP data layers
Implement IAM, least-privilege access, encryption, service accounts, network security, and data access policies
Work with governance and security teams to ensure compliance requirements are met
Lead infrastructure automation using Terraform
Build repeatable and secure GCP infrastructure deployments
Implement CI/CD pipelines for data engineering workloads
Automate data pipeline deployment, testing and infrastructure provisioning
Establish Dev, QA, UAT and Production deployment standards
Lead performance optimization initiatives across GCP data workloads
Optimize BigQuery query performance, partitioning and clustering, Dataflow pipelines, Spark workloads, Cloud Storage, and streaming workloads
Analyze AWS workloads and determine the most cost-effective GCP architecture
Develop cloud FinOps and cost optimization strategies
Establish performance benchmarks and SLAs for critical workloads
Lead and mentor a team of Data Engineers, Senior Data Engineers and Technical Leads
Provide technical direction and establish engineering standards
Conduct architecture and code reviews
Define technical roadmaps and engineering priorities
Break complex migration requirements into actionable deliverables
Track engineering progress, risks, dependencies and delivery milestones
Promote best practices around coding, testing, CI/CD, security and documentation
Mentor engineers on GCP, data architecture and modern data engineering practices
Act as the primary technical point of contact for US-based stakeholders
Work closely with Business, Product, Data Science, BI and DevOps teams
Translate business requirements into scalable technical solutions
Present architecture decisions, migration strategies and technical roadmaps
Communicate technical risks, dependencies, timelines and trade-offs
Collaborate with business teams to define operational and analytical KPIs
Design and implement AI/ML and Generative AI solutions on GCP using Vertex AI, BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Run, and related GCP-native services
Build production-grade machine learning pipelines for data preparation, model training, validation, evaluation, deployment, monitoring, retraining, and lifecycle management
Develop Generative AI and Retrieval-Augmented Generation (RAG) solutions, including enterprise search, document intelligence, AI assistants, summarization, semantic search, embeddings, vector search, and knowledge-management applications
Design scalable ingestion, transformation, chunking, embedding, indexing, and retrieval pipelines for structured and unstructured enterprise data
Implement MLOps practices using Vertex AI Pipelines, Model Registry, model endpoints, Terraform, GitHub, Cloud Build, and CI/CD pipelines
Establish standards for model versioning, experiment tracking, data and feature validation, automated testing, deployment approvals, rollback, and environment promotion
Implement monitoring for model performance, data drift, latency, reliability, inference cost, response quality, retrieval accuracy, and GenAI risks such as hallucination and prompt injection
Ensure responsible AI, data privacy, security, governance, access control, auditability, and human-review processes are incorporated into AI/ML and GenAI solutions
Partner with Data Science, Analytics, Product, BI, Security, and US-based stakeholders to identify, prioritize, and deliver high-value AI/ML and GenAI use cases

Requirements:

Experience: 15+ Years
Location: Remote (USA)
The candidate should have strong hands-on experience with AWS services such as S3, Glue, Redshift, Athena, Step Functions and AWS DMS, along with strong GCP expertise across BigQuery, Dataflow, Pub/Sub, Cloud Storage and Cloud Composer
15+ years of experience in Data Engineering, Data Architecture, Cloud Engineering, AI/ML Engineering, or related technology leadership roles
Strong hands-on experience with AWS Data Engineering and Data Architecture
5+ years of strong hands-on GCP Data Engineering experience
Proven experience delivering AWS-to-GCP migration projects
Strong experience designing enterprise Data Lake and Lakehouse platforms on GCP
Strong hands-on experience with BigQuery, Google Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, Dataproc, IAM, and Terraform
Experience migrating AWS data workloads, pipelines, and platforms to GCP
Strong knowledge of AWS and GCP service mapping, migration patterns, modernization strategies, and cloud architecture best practices
Experience designing, building, and deploying AI/ML solutions on GCP using Vertex AI
Hands-on experience with Generative AI, LLM-based applications, RAG architectures, embeddings, vector search, prompt engineering, and enterprise AI assistants
Strong understanding of MLOps, including model training, model registry, CI/CD/CT, model deployment, monitoring, retraining, governance, and rollback strategies
Experience implementing secure and responsible AI solutions, including data privacy, model evaluation, access controls, auditability, and governance
Expert-level SQL and strong Python and PySpark skills
Strong data modeling, data warehousing, batch processing, and real-time data engineering experience
Experience with Terraform, Git, GitHub, Cloud Build, CI/CD pipelines, and infrastructure automation
Experience managing and mentoring data engineering and cross-functional technical teams
Strong communication skills with experience working with US-based stakeholders
AWS – Strong Existing Platform Experience
Amazon S3
AWS Glue
AWS Glue Data Quality
Amazon Redshift / Redshift Serverless
Amazon Athena
AWS Step Functions
AWS DMS
AWS Lake Formation
IAM
GCP – Target Platform
BigQuery
Google Cloud Storage
Pub/Sub
Dataflow / Apache Beam
Cloud Composer / Airflow
Dataproc / Spark
Cloud Monitoring
Cloud Logging
Dataplex / Data Catalog
Advanced Python
Expert-level SQL
Strong PySpark / Apache Spark
ETL/ELT
CDC
Batch and streaming data processing
Event-driven architecture
Data pipeline optimization
Enterprise Data Lake / Lakehouse
Medallion Architecture
Data Modeling
Dimensional Modeling
Multi-tenant Data Modeling
Schema-on-Read / Schema-on-Write
Data Lineage
Metadata Management
Data Governance
dbt
Apache Airflow
Terraform
Git / GitHub
Cloud Build
CI/CD
Data Quality Frameworks
Experience with Dataplex, Data Catalog and data lineage is preferred
Google Cloud Professional Data Engineer certification
Google Cloud Professional Machine Learning Engineer certification
Experience with Vertex AI Agent Builder, Vertex AI Search, Gemini models on Vertex AI, or enterprise Generative AI platforms
Experience with dbt, Apache Airflow, Kafka, Apache Spark, Kubernetes, Cloud Run, and API-driven architectures
Experience with Dataplex, Data Catalog, data lineage, metadata management, data governance, master data management, and data-quality frameworks
Experience supporting enterprise or regulated environments with strong data privacy, security, compliance, audit, and governance requirements
OpenLineage is a plus