Data Engineer - Data Platform

Kraken is a mission-focused company rooted in crypto values, seeking a skilled Data Engineer to join their Data Platform team. The role involves building and maintaining high-performance data pipelines, ensuring data reliability and scalability, and enabling access to clean and timely data across the organization.

Responsibilities:

• Build scalable and reliable data pipelines that collect, transform, load and curate data from internal systems
• Augment data platform with data pipelines from external systems
• Ensure high data quality for pipelines you build and make them auditable
• Drive data systems to be as near real-time as possible
• Support design and deployment of distributed data store that will be central source of truth across the organization
• Build data connections to company's internal IT systems
• Develop, customize, configure self service tools that help our data consumers to extract and analyze data from our massive internal data store
• Evaluate new technologies and build prototypes for continuous improvements in data engineering

Requirements:

• 5+ years of work experience in relevant field (Data Engineer, DWH Engineer, Software Engineer, etc)
• Experience with data-lake and data-warehousing technologies and relevant data modeling best practices (Presto, Athena, Glue, etc)
• Proficiency in at least one of the main programming languages used: Python and Scala. Additional programming languages expertise is a big plus!
• Experience building data pipelines/ETL in Airflow, and familiarity with software design principles
• Excellent SQL and data manipulation skills using common frameworks like Spark/PySpark, or similar
• Expertise in Apache Spark, or similar Big Data technologies, with a proven record of processing high volumes and velocity of datasets
• Experience with business requirements gathering for data sourcing
• Bonus - Kafka and other streaming technologies like Apache Flink