Key skills
PythonPySparkSparkSnowflakeDatabricksCloud-based data platformsData pipeline designBatch data processingStreaming data processingMedallion ArchitectureAutoscalingCluster optimizationCost-efficient data processingCI/CDDevOpsVersion controlTerraformInfrastructure-as-codeAIAnalyticsVersion Control
About this role
Qualitest is the World’s leading AI-powered Quality Engineering Company, and they are seeking a Data Engineer to join their growing team in the United States. The role involves collaborating with various teams to design reliable and scalable data solutions while working with modern data management platforms and data pipeline design.
Responsibilities:
- Strong hands-on experience in Python and PySpark, with a solid understanding of Spark concepts, performance tuning, and scalable data processing
- Experience working with modern data management platforms such as Snowflake, Databricks, or similar cloud-based data platforms
- Good understanding of data pipeline design, development, orchestration, and monitoring, including batch and streaming data processing
- Familiarity with Medallion Architecture concepts, including bronze, silver, and gold layer design and best practices for data transformation
- Strong understanding of autoscaling, cluster optimization, and cost-efficient data processing in cloud or distributed data environments
- Experience with CI/CD, DevOps practices, and version control, with preference for candidates who have exposure to Terraform or infrastructure-as-code tools
- Ability to collaborate with business, analytics, and engineering teams to design reliable, reusable, and scalable data solutions
Requirements:
- Strong hands-on experience in Python and PySpark, with a solid understanding of Spark concepts, performance tuning, and scalable data processing
- Experience working with modern data management platforms such as Snowflake, Databricks, or similar cloud-based data platforms
- Good understanding of data pipeline design, development, orchestration, and monitoring, including batch and streaming data processing
- Familiarity with Medallion Architecture concepts, including bronze, silver, and gold layer design and best practices for data transformation
- Strong understanding of autoscaling, cluster optimization, and cost-efficient data processing in cloud or distributed data environments
- Experience with CI/CD, DevOps practices, and version control, with preference for candidates who have exposure to Terraform or infrastructure-as-code tools
- Ability to collaborate with business, analytics, and engineering teams to design reliable, reusable, and scalable data solutions