Senior Data Engineer
Minneapolis, Minnesota, United States of America
Full Time
1 hour ago
Key skills
ApacheETLPySparkScalaSparkSQLELTData EngineeringData LakeAnalyticsDatabricksApache SparkGitVersion Control
About this role
Role Overview
- Design, develop, and maintain scalable data pipelines using Apache Spark (PySpark and/or Scala)
- Build and optimize data workflows on Databricks , including Delta Lake, notebooks, and scheduled jobs
- Ingest, transform, and curate large-scale structured and semi-structured datasets
- Perform performance tuning and cost optimization of Spark workloads and Databricks clusters
- Implement data quality checks, monitoring, and error handling
- Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
- Support batch processing and, where applicable, streaming data pipelines
- Follow best practices for testing, documentation, security, and version control
Requirements
- 5+ years of experience in Data Engineering or related roles
- Strong hands-on experience with Apache Spark (PySpark or Scala)
- Proven experience working in Databricks environments
- Strong SQL skills and experience with relational and analytical databases
- Experience building and maintaining ETL/ELT pipelines at scale
- Familiarity with modern data lake architectures (Delta Lake preferred)
- Experience with Git-based version control
Tech Stack
- Apache
- ETL
- PySpark
- Scala
- Spark
- SQL
Benefits
- Flexible work from home options