Key skills
PythonSQLMachine LearningETLData EngineeringAnalyticsBusiness IntelligenceDatabricksSparkAWSDockerJenkinsPostgreSQLGitSaaSJiraAgileCommunicationCollaboration
About this role
E Source is a research, data/analytics, and technology focused professional services firm specializing in the utility industry. They are seeking a Databricks Data Engineer to design, build, and maintain data pipelines and systems that support consulting services and SaaS solutions, collaborating with cross-functional teams to deliver effective data products for clients.
Responsibilities:
- Design, develop, and maintain data pipelines, infrastructure, and systems using AWS, Python, Databricks, Spark, and PostgreSQL
- Collaborate with cross-functional teams to translate business requirements into technical solutions
- Provide technical guidance and mentorship to junior data engineers
- Develop and implement data engineering standards and best practices
- Monitor and troubleshoot data pipelines to ensure data quality, integrity, and availability
- Research industry trends and evaluate new data technologies
- Build and maintain data lakes to support analytics and business intelligence
- Manage the software development life cycle and DevOps activities
- Partner with data scientists and analysts to deliver effective data solutions
- Develop data validation and data analysis tools
Requirements:
- Bachelor's degree in computer science, information technology, or a related field
- Five or more years of experience in data engineering or a related role
- Expert proficiency in Python, SQL, Databricks, and Spark
- Experience building cloud-based data pipelines in AWS
- Knowledge of data governance, security, privacy, and retention practices
- Strong communication, collaboration, and problem-solving skills
- Experience working in agile environments using tools such as Jira
- Proficiency with Git, Docker, and CI tools such as Jenkins
- Passion for data and continuous improvement
- Ability to work independently and proactively
- Experience optimizing ETL workflows in Databricks
- Knowledge of data lineage and data quality practices
- Experience supporting data science and machine learning solutions
- Prior experience in the energy or utility industry
- Master's degree in a relevant STEM field