HYR Global Source Inc is seeking a Data Engineer experienced in building and managing data pipelines using modern data platforms and cloud technologies. The role involves hands-on development, data processing, and automation across large-scale data systems.
Responsibilities:
- Design, build, and maintain scalable ETL/data pipelines in the cloud
- Work with both batch and streaming data for analytics and reporting
- Develop and optimize data transformations using Python, SQL, and Spark
- Collaborate with team members to ensure pipeline reliability, performance, and quality
- Support visualization and reporting through BI tools
Requirements:
- Programming: Python, SQL
- Data Processing: Apache Spark, Databricks, Azure Data Factory
- Streaming: Kafka, Pub/Sub
- DevOps/Automation: Git, Jenkins/GitHub Actions, Azure DevOps, Terraform, Docker
- Visualization: Power BI, Tableau, or similar tools
- Experience with data lake or warehouse architectures
- Familiarity with data governance, testing, or CI/CD best practices