GCP Data Engineer

CoSourcing Partners is an Enterprise-AI and IT Services Company seeking a highly skilled GCP Data Engineer. The role involves designing, developing, and optimizing scalable data solutions on Google Cloud Platform, focusing on building robust data pipelines and ensuring data quality for analytics and machine learning use cases.

Responsibilities:

• Design, build, and optimize scalable batch and real-time (streaming) data pipelines using GCPnative services
• Develop and maintain data ingestion frameworks leveraging tools such as Pub/Sub, Dataflow, and Cloud Storage
• Implement data transformation pipelines using BigQuery, dbt, and Python-based workflows
• Ensure efficient handling of large-scale structured and unstructured datasets
• Design and implement high-performance data models for cloud-based data lakes, data warehouses, and analytics platforms
• Optimize data schemas and partitioning strategies in BigQuery for performance and cost efficiency
• Support modern architectures such as medallion (bronze/silver/gold) layers and lakehouse patterns
• Write advanced SQL queries for transformation, validation, and analytics
• Develop scalable data processing logic using Python and/or Apache Beam
• Build reusable, modular, and maintainable code for data workflows
• Implement and maintain data quality checks, validation rules, and anomaly detection frameworks
• Enable data observability through monitoring, logging, and alerting mechanisms
• Ensure highly reliable data pipelines with fault tolerance and error handling strategies
• Support migration and modernization efforts from legacy ETL tools (e.g., Talend) to GCP-native ELT frameworks (dbt)
• Optimize existing pipelines for performance, scalability, and maintainability in cloud environments
• Drive adoption of ELT best practices using BigQuery as the compute engine
• Collaborate with data architects, business analysts, and machine learning teams to deliver trusted datasets
• Translate business requirements into scalable data solutions
• Provide technical guidance and support for downstream analytics and reporting use cases
• Drive adoption of best practices in cloud data engineering, CI/CD, and DevOps
• Implement secure data access controls using IAM roles, policies, and governance frameworks
• Follow standards for code quality, version control (Git), and automated deployments

Requirements:

• Bachelor's or Master's degree in Computer Science, Engineering, or related field
• 4+ years of experience in data engineering or data platform development
• Hands-on experience with Google Cloud Platform (GCP) services: BigQuery, Dataflow, Pub/Sub, Cloud Storage
• Strong proficiency in SQL and Python
• Experience with dbt (Data Build Tool) or similar ELT frameworks
• Experience building batch and streaming data pipelines
• Experience with Apache Beam or Spark
• Familiarity with Talend or other ETL tools and migration to cloud-native solutions
• Knowledge of data lakehouse architectures and modern data stack
• Experience with CI/CD tools (e.g., GitHub Actions, Cloud Build, Jenkins)
• Understanding of data security, governance, and compliance standards
• Exposure to machine learning data pipelines and feature engineering