Key skills
AzureDatabricksApache SparkSQLDBTPythonApache AirflowData EngineeringData ArchitectureETL/ELT DevelopmentETLELTSparkAirflowdbtLeadership
About this role
Dice is seeking an experienced Senior Data Engineer with over 10 years of experience in data engineering and software engineering environments. The ideal candidate will have extensive expertise in designing and delivering scalable, high-performance data solutions using Azure, Databricks, Spark, SQL, DBT, Python, and Airflow, while providing strong technical leadership and architecture experience.
Responsibilities:
- Design, develop, and implement scalable data engineering solutions in cloud-based environments
- Lead end-to-end data engineering initiatives and provide technical direction to project teams
- Architect and optimize large-scale data processing frameworks and pipelines
- Develop and maintain reusable data engineering frameworks and best practices
- Build robust ETL/ELT workflows using modern data engineering technologies
- Collaborate with business stakeholders to translate requirements into technical solutions
- Ensure data quality, performance, security, and governance standards are maintained
- Optimize data models, processing jobs, and analytical workloads for efficiency and scalability
- Mentor junior engineers and provide technical leadership across teams
- Support production deployments, troubleshooting, and continuous improvement initiatives
Requirements:
- 10+ years of experience in software engineering and data engineering environments
- 7+ years of experience leading data engineering initiatives
- 7+ years of technical leadership and data architecture experience
- 7+ years of hands-on experience with Apache Spark
- 7+ years of hands-on experience with Databricks
- 7+ years of hands-on experience with DBT
- 7+ years of hands-on experience with Python
- 7+ years of hands-on experience with Apache Airflow
- Strong experience in designing scalable data platforms and data pipelines
- Expertise in data modeling, ETL/ELT development, and data integration
- Advanced SQL programming and query optimization skills
- Experience working with Azure cloud services
- Strong problem-solving and analytical skills
- Experience building enterprise-scale data platforms
- Expertise in developing reusable frameworks and accelerators for data engineering
- Experience working in complex, data-driven enterprise environments
- Knowledge of cloud-native architecture and modern data lakehouse solutions
- Experience with performance tuning and optimization of big data workloads