Senior Data Engineer

Sportradar is the world’s leading sports technology company, at the intersection between sports, media, and betting. They are seeking a Senior Data Engineer to design and implement large-scale data pipelines and ensure efficient data processing and analytics for their Sport Performance products.

Responsibilities:

• Design, build, and implement generalized large-scale, sophisticated data pipelines using Nifi for downstream analytics and data science for our Sport Performance products
• Design and develop scalable Nifi ingestion pipelines within AWS cloud services to consume real-time and batch data from external sources
• Ensure the seamless integration of AWS-based tools for data storage, processing, and analytics
• Responsible for ETL development and warehousing using Python and Java
• Create data pipeline triggers and filters within ETL (extract, transform, and load) process to ensure appropriate optimization of data flowing through system and resource usage
• Implement monitoring and error handling for all new parts of the data pipeline to ensure observability and alerting is available
• Establish rigorous unit testing across the data pipeline to ensure robustness of the system
• Design and create data models for use throughout the ETL system
• Utilize Kafka to efficiently and to effectively store data to move throughout the data pipeline and for downstream data science and analytics usage
• Design data architecture and data models for both internal and external representations of data
• Build the data transforms within the data pipeline to convert data from external to internal representations
• Conduct data analytics and debugging of bad data by writing SQL queries
• Build automated cleaning of data to remove bad or unusable data from downstream consumers with logging to understand the frequency and depth of the underlying issues
• Collaborate with other engineering teams to adopt standard methodologies, drive scalability, and increase consistency across systems
• Maintain awareness of company standards and technology guidance; use JIRA, an Agile project mgmt. tool, to ensure efficient data development; collaborate with peers to align projects with overall direction
• Follow best practices across Data Engineering to ensure scalable, consistent data architecture and system
• Utilize Java language to build data processor in Nifi framework
• Utilize Docker to ensure consistent, repeatable, and isolated environment for software development and testing
• Work in a self-driven, independent fashion to meet Sport driven deadlines

Requirements:

• Master's degree in Computer Science, Computer Engineering, or closely related field and 1 year experience as a data engineer or related occupation
• 1 year of experience with Python, Java, Kafka, AWS, and Docker
• ETL Development and Warehousing
• Analytic and debugging using SQL
• Agile development environment
• Designing data architecture