Design and build robust data pipelines and feature stores that powers our AI products.
Analyse data requirements, understand complex data sources, and partner with architects to determine the best methods to extract, transform, and load data into data platform.
Lead conversations with business or functional teams to translate user requirements into reusable and cost-efficient data acquisition methodologies for AI products.
Be the key anchor for data extraction, preparation and hosting processes representing the AI team. Data expert for engineering datasets used by data scientists working on various use cases.
Perform functional and stress testing on data models and extraction jobs to ensure that design is optimized for performance, scale, and reusability.
Manage foundational data administration tasks such as scheduling jobs, troubleshooting job errors, identifying issues with job windows, assisting with database backups and performance tuning.
Ensure data quality throughout entire development process, including audits and feedback loops to sources of truth.
Create or update technical documentation for transition to support teams.
Mentor junior data engineers
Requirements
B.S. in Computer Science or related field
5+ years of experience in data engineering
Experience with advanced analytics applications
Strong understanding of data quality processes
Experience in working with business or functional teams
Ability to work on problems impacting multiple business units
Familiarity with big data technologies
Experience in a global technology environment
Benefits
A comprehensive benefits package including health insurance