Senior Data Engineer

The Walt Disney Company is known for creating world-class immersive digital experiences. They are seeking a Senior Data Engineer to build and operate a near-real-time data streaming platform to enhance guest experiences by delivering data products efficiently and reliably.

Responsibilities:

• Contribute to the design, building, and supervision of technology architecture, solutions, and software dedicated to capturing, managing, and using both structured and unstructured data from various sources
• Develop efficient processes and structures to streamline data flow, routing, and storage, adhering to both business and technical specifications, and leveraging cloud and local storage options as required
• Apply technical tools and programming skills for the purposes of data cleansing, organization, transformation, and upkeep, integrating automation and advanced methodologies such as big data, artificial intelligence, and machine learning
• Uphold design standards and quality assurance protocols for the development of software and systems, ensuring seamless data compatibility, functionality, and integrity across all data connections, transmissions, and storage platforms
• Evaluate internal and external business and product requirements to ensure alignment of data operations and endeavors with overarching organizational objectives and priorities
• Develop data pipelines and shared monitoring infrastructure that power observability, pipeline health, and data quality for the broader team
• Create infrastructure data products (cost tracking, resource utilization, audit logs)
• You will be expected to stay up to date with emerging technologies
• May be required to provide support escalation, oversee the ongoing operations, and support the enhancement of existing tools as needed

Requirements:

• Minimum 5 years of Data Engineering, Software Development or similar experience required
• Proficiency in fundamentals of data pipelining, ELT/ETL processes, data architecture, and the overall data lifecycle, complemented by hands-on experience in designing and implementing efficient data workflows and processes
• Demonstrating ability to contribute across multiple phases of development for complex projects, including technical design, implementation, and end-to-end testing, ensuring the successful delivery of data engineering solutions
• Passionate about delivering data engineering projects collaboratively, with a strong commitment to achieving team goals
• Effective troubleshooting skills, capable of identifying and resolving sophisticated issues with sound judgment and initiative
• Comfortable working in an agile environment, with prior experience in Agile and Scrum methodologies, ensuring adaptability to evolving project requirements and effective collaboration with cross-functional teams
• Proficiency in SQL, Python, AWS services (e.g. IAM, S3, Kinesis Data Stream, Kinesis Firehose, Lambda, DynamoDB, DMS, EventBridge), with a proactive demeanor towards continuous learning and skill development, and an interest in mastering emerging data engineering tools and methodologies
• Experience working across both relational and NoSQL databases (e.g., Snowflake, DynamoDB, Redis, MongoDB), with the ability to optimize data structures and storage choices for performance and scalability
• Bachelor's degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience required
• Expertise coding and developing solutions that solve complex problems, while keeping in mind code space and time complexity
• Proficiency in DevOps practices, e.g.: CI/CD, docker, testing, and automation
• Understanding of trade-offs between different data storage systems and architectures (data warehouses, SQL vs NoSQL, partitioning, etc.)
• Knowledge of DevOps concepts
• Experience with API development (e.g., RESTful services, API design, authentication) for exposing and consuming data products and services resiliently, performantly, and at scale
• Experience developing real-time data processing using Apache Flink, or comparable stream-processing frameworks (e.g., Spark, Kafka, or Apache Beam)
• Experience with infrastructure-as-code, e.g.: Terraform, CloudFormation, AWS CDK, etc