Compunnel Inc.Remote
Big Data Engineer(ONLY W2) -- CHADC5785402
United States of America
Full Time
3 days ago
No H1B
Key skills
PythonScalaSQLMachine LearningBigQuerySparkKafkaApache KafkaAirflowGCPGitLab CIRESTfulGitLabCI/CD
About this role
Compunnel Inc. is seeking a Big Data Engineer to work with their cloud-based data solutions. The role focuses on developing and managing data pipelines using technologies such as Python, SQL, and GCP.
Requirements:
- 5-10 years of experience
- Python
- Airflow
- PySpark
- Spark SQL
- Spark Streaming
- Scala
- Relational/No SQL databases
- Apache Kafka
- writing complex sql queries
- GCP (Bigquery, Dataproc)
- working with RESTful APIs in data pipelines
- CI/CD (gitlab CI)
- GCP Shop - using Big Query
- Does NOT want a machine learning engineer, wants a big data engineer