Research, evaluate, and experiment with emerging data engineering technologies in a lab environment to stay ahead of industry trends.
Collaborate with data engineering teams to demonstrate the capabilities of new technologies and facilitate their adoption.
Help define and refine processes and best practices for the data engineering discipline.
Partner with business stakeholders, data architects, ETL developers, reporting teams, data scientists, and IT counterparts to identify and integrate data from various sources including external systems, internal databases, and data warehouses.
Develop, test, deploy, monitor, and troubleshoot data pipelines and automation processes.
Maintain comprehensive documentation and ensure reconciliation of data processes.

Bachelor's degree in related field or equivalent work experience.
2-4 years of hands-on experience in data engineering.
2-4 years of experience processing large datasets using tools such as Databricks, Python, Spark, Kafka, and APIs.
Proficiency in ingesting diverse data formats (XML, JSON, Parquet, SequenceFile) and integrating with various data sources (Oracle, cloud databases, Event Hub, Service Bus).
Experience with cloud platforms (Azure, AWS) and infrastructure automation tools (ARM Templates, Terraform, CloudFormation).
Strong understanding of cloud computing models (IaaS, PaaS, SaaS) and hybrid cloud environments.
Familiarity with object storage technologies (Data Lake Storage Gen2, Blob, S3, ADLS).
Experience with Agile/Scrum methodologies and application lifecycle management.
Proficient in source control (Git), CI/CD pipelines (Azure DevOps), and artifact repositories (Artifactory).
Experience creating and maintaining ETL processes.
Knowledge of IT governance, privacy compliance, and GDPR.
Preferred experience with Adobe Experience Platform, REST APIs, IBM Infosphere (Datastage), and Unix scripting.

Generous benefits package available on day one to include: 401K matching, bonding leave for new parents (12 weeks, 100% paid), tuition assistance, training, GM employee auto discount, community service pay and nine company holidays.

Data Engineer II

Key skills