Data Engineer II

Dataiku is the Platform for AI Success, focusing on enterprise orchestration for building and deploying AI. The Data Engineer II will play a crucial role in delivering data for analytics and AI, contributing to the Data Platform and collaborating with various teams to ensure engineering excellence.

Responsibilities:

• Be an expert level engineer within the Dataiku Platform including Platform Automation, GenAI Capabilities, Plugin Development, maintenance & troubleshooting
• Be an expert level engineer within Snowflake for data engineering and security/governance features
• Build & maintain python & SQL based platform automation process
• Build & maintain data quality metrics & observability to help drive data quality standards
• Design data models for both short term and long term use cases to support data warehouse scalability
• Build & maintain administration systems and applications for monitoring, alerting, data observability, access management, platform metrics, and end user transparency
• Build & maintain GenAI Platform platform solutions focused on security and governance for engineering delivery
• Build & maintain DataOps process for SDLC delivery
• Identify opportunities for improvements & optimization for greater scalability & delivery velocity
• Collaborate closely with Analytics Engineers to provide data & data models for analytical deliverables
• Perform root cause analysis on often complex errors to help ensure data pipeline availability
• Help drive technical & architectural decisions on the data platform including decisions on data architecture, data engineering processes, data quality frameworks, data access security & governance frameworks, DataOps processes & data consumption models
• Help test new features in Dataiku and partner tools to both provide feedback internally as well as determine value towards internal analytics & data platform integration
• Work closely with key stakeholders across the organization including Infra, embedded analytics teams, Product and Engineering to help foster both technical implementations & requirements gathering
• Proactively drive innovation internally with dedicated innovation time & projects that aim to be transformational for either the platform, team or company as a whole
• Actively contribute to the expertise level and competencies of the EDA Team and participate in the creation and support of data development standards and best practices

Requirements:

• 3+ years of relevant experience in Data Engineering / Data Platform Engineering
• Expertise in SQL & Python is a must. Experience in Dataiku DSS is a big plus
• Prior experience with Snowflake strongly desired
• Prior experience with DevOps technologies such as Github Actions, Azure DevOps or Jenkins
• Strong understanding of data architecture & data modeling concepts
• Prior experience building and maintaining replication & data pipelines in a cloud data warehouse or data lake environment
• Excellent analytical and creative problem-solving skills - exhibit confidence to ask questions to bring clarity, share ideas and challenge the norm
• Passion for continuous learning and teaching to help learn & teach new technologies & implementation strategies
• Experience working with complex stakeholders; dissecting vague asks and helping to define tangible requirements
• Ability to manage multiple projects and time constraints simultaneously in a high trust remote environment
• Ability to wear multiple hats depending on the project with the focus on accomplishing end goals while inspiring colleagues to do the same
• Excellent written and verbal communication skills (especially with senior level stakeholders) with the ability to speak to both the business value, data products, & technical capabilities of a platform. Ability to create clear and concise documentations with a high degree of precision