Key skills
ETLNoSQLPythonSparkSQLMachine LearningData EngineeringAnalyticsDatabricksGitHubCI/CD
About this role
Role Overview
- Leads one or more project teams of other data engineers for all stages of design and development for complex, secure, and performant data solutions and models
- Reviews and evaluates designs and project activities for compliance with architecture, security, and quality guidelines and standards
- Creates and maintains optimal data pipeline/ETL architecture
- Collaborates with businesses, application and process owners, and product team members to define requirements and design solutions for the company’s big data and analytics solutions
- Builds the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Databricks and other technologies
Requirements
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent
- 3-7 years’ experience
- Databricks for data engineering, analytics, and machine learning workflows
- Python and SQL programming and advanced ETL pipelines
- Lakeflow and Spark-based big data processing and orchestration on Databricks
- GitHub, CI/CD, Databricks Asset Bundles, and modern software engineering practices applied to data engineering
- Experience with NoSQL and relational systems
Tech Stack
- ETL
- NoSQL
- Python
- Spark
- SQL
Benefits
- Hybrid/Flexible work culture