Diverse LynxRemote
GCP Cloud Engineer with Java
United States of America
Full Time
3 weeks ago
H1B Sponsor Likely
Key skills
GCP ComposerApache AirflowGoogle Cloud StorageBigQueryCloud DataflowApache BeamCloud DataprocCloud FunctionsCloud RunIAMService account permissionsSecret ManagerStackdriverCloud MonitoringCloud LoggingRDBMSPLSQLSpringbootDockerfileTerraformCI/CD pipelinesJenkinsJavaPythonDNSSSO configurationAirflowAngularGCPGoogle CloudCloud StorageGCSSSOcomposerCI/CDCommunication
About this role
Diverse Lynx is a company that promotes and supports a diverse workforce. They are seeking a GCP Cloud Engineer with Java expertise to enhance and manage data ingestion pipelines, monitor GCP infrastructure, and collaborate with DevOps teams for CI/CD processes.
Responsibilities:
- Fix and enhance Dataflow and/or Dataproc Data ingestion pipelines (batch & Streaming)
- Have Good Understanding of GCP Composer (Apache Airflow) environments
- Manage Airflow DAGs and troubleshoot Airflow-related issues
- Monitor and optimize Composer performance and scalability
- Maintain GCP infrastructure including services like: Google Cloud Storage (GCS), BigQuery (BQ), Cloud Dataflow (Apache Beam), Cloud Dataproc, Cloud Functions, Cloud Run
- Have Good Understanding of: IAM and service account permissions, Secret Manager
- Monitor GCP usage and quotas, and proactively prevent quota-related issues
- Set up and manage alerting, logging, and monitoring using tools like Stackdriver (Cloud Monitoring & Logging)
- DB design experience (with Excellent RDBS-PLSQL knowledge)
- Design Springboot APIs
- Analyze, design, create Dockerfile
- Knowledge on Angular (Good to Have)
- Design and implement infrastructure as code using Terraform for provisioning and managing GCP resources
- Collaborate with DevOps and development teams to manage CI/CD pipelines using Jenkins and other automation tools
- Ensure security, compliance, and cost-efficiency across the cloud environment
- Document system configurations, procedures, and best practices
- Proficient in either Java or Python programming languages
- Excellent problem-solving and troubleshooting skills
- Strong communication and documentation abilities
- Knowledge on DNS and SSO configuration setup
- Knowledge on CDC Pipelines/Application (Good to Have)
Requirements:
- GCP Composer (Apache Airflow)
- Fix and enhance Dataflow and/or Dataproc Data ingestion pipelines (batch & Streaming)
- Good Understanding of GCP Composer (Apache Airflow) environments
- Manage Airflow DAGs and troubleshoot Airflow-related issues
- Monitor and optimize Composer performance and scalability
- Maintain GCP infrastructure including services like Google Cloud Storage (GCS), BigQuery (BQ), Cloud Dataflow (Apache Beam), Cloud Dataproc, Cloud Functions, Cloud Run
- Good Understanding of IAM and service account permissions
- Good Understanding of Secret Manager
- Monitor GCP usage and quotas, and proactively prevent quota-related issues
- Set up and manage alerting, logging, and monitoring using tools like Stackdriver (Cloud Monitoring & Logging)
- DB design experience (with Excellent RDBS-PLSQL knowledge)
- Design Springboot APIs
- Analyze, design, create Dockerfile
- Design and implement infrastructure as code using Terraform for provisioning and managing GCP resources
- Collaborate with DevOps and development teams to manage CI/CD pipelines using Jenkins and other automation tools
- Ensure security, compliance, and cost-efficiency across the cloud environment
- Document system configurations, procedures, and best practices
- Proficient in either Java or Python programming languages
- Excellent problem-solving and troubleshooting skills
- Strong communication and documentation abilities
- Knowledge on DNS and SSO configuration setup
- Knowledge on Angular ( Good to Have )
- Knowledge on CDC Pipelines/Application (Good to Have)