Key skills
BigQueryETLGoogle Cloud PlatformMongoDBNoSQLPySparkPythonScalaSparkSQLMLELTData EngineeringData LakeAnalyticsGCPGoogle CloudCommunication
About this role
Role Overview
- Build a centralized data lake on GCP by integrating diverse data sources
- Develop, maintain, and optimize SPARK-powered data processing pipelines
- Design and implement data validation and quality checks
- Collaborate with Data Science and ML teams for analytics
- Collaborate with cross-functional teams for data insights
- Design, develop, and maintain data dashboards
- Manage ETL/ELT processes and troubleshoot data issues
Requirements
- 5+ years of experience in data engineering
- Proficient in PySpark for batch and streaming
- Strong skills in GCP services (Dataproc, Dataflow, BigQuery)
- NoSQL experience, preferably with MongoDB
- Programming skills in Scala/Python
- Understanding of Data Governance and Data Modelling
- Good SQL knowledge
- Effective communication skills
Tech Stack
- BigQuery
- ETL
- Google Cloud Platform
- MongoDB
- NoSQL
- PySpark
- Python
- Scala
- Spark
- SQL