Data Engineer

Role Overview

• Responsible for acquiring, curating, and publishing data both on prem and in the cloud for analytical or operational uses for basic to moderate scenarios.
• Ensures the data is in a ready-to-use form that creates a single version of the truth across all data consumers, including business/technology users, reporting and visualization specialists and data scientists with coaching and support.
• Utilizes skills to translate business analytic requests/requirements into design, development, testing, deployment, and production maintenance tasks.
• Works with various technologies from big data, relational and non-relational databases, cloud environments, different programming languages and various reporting tools and is familiar with a few but requires training for some.
• Consults on data projects of intermediate complexity, with limited coaching and guidance, to understand design requirement and existing business processes to lead in the design, development, and implementation of data products.
• Practices code management and integration with current architectural and data governance guidelines with reduced oversight required of leadership.
• Produces data building blocks, data models, and data flows for varying client requests such as dimensional data, standard and ad hoc reporting, data feeds, dashboard reporting, and data science research and exploration with limited assistance and coaching.
• Creates business user access methods to structured and unstructured data.
• Utilizes techniques such as mapping data to a common data model, natural language processing, transforming data as necessary to satisfy business rules, AI, statistical computations, and validation of data content.
• Translates business data stories into a technical story breakdown structure and work estimates for a schedule or planned agile sprint.
• Projects are of increasing complexity but require less coaching and assistance from leadership.
• Develops and maintains moderately complex scalable data pipelines for both streaming and batch requirements and builds out new API integrations to support increased demands of data volume and complexity.
• Preps/cleanses data to optimize for downstream reporting via Farmers standard visualization or AI/ML tools with coaching and feedback.
• Applies sound knowledge of data governance concepts with limited experience in application.
• Executes on mid size projects and large tasks within timelines.
• Develops data project plans and executes with some supervision.
• Performs other duties as assigned.

Requirements

• Strong SQL skills
• dbt Core or dbt Cloud (macros, Jinja, packages, exposures, tests)
• Airflow DAGs, ideally in an Astronomer environment
• cloud data warehouse
• Snowflake, BigQuery, or Redshift
• Comfort working in Python, at minimum for writing Airflow operators and basic scripting
• Experience with Git-based workflows and CI/CD for data pipelines

Tech Stack

• Airflow
• Amazon Redshift
• BigQuery
• Cloud
• Python
• SQL

Benefits

• Competitive salary and performance-based bonuses
• Comprehensive benefits package
• Career development and training opportunities
• Flexible work arrangements (remote and/or office-based)
• Dynamic and inclusive work culture within a globally renowned group
• Private Health Insurance
• Pension Plan
• Paid Time Off
• Training & Development