Senior Software Development Engineer, Big Data

Zillow is a leading real estate platform in the U.S. seeking a Senior Software Development Engineer to join their Housing Trends Data Engineering team. The role involves processing large datasets to produce housing market insights and collaborating with data science teams to enhance data products and processing frameworks.

Responsibilities:

• Build data systems to link raw data and concrete Housing Trends insights
• Build and extend our internal forecasting model development framework
• Develop and maintain scalable data products and deliver pipelines built for speed, accuracy, and consistency that will scale as our stakeholders’ need
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability and performance, etc
• Collaborate with data scientists to understand their data requirements and build systems/tools that enable efficient model training and experimentation
• Implement feature stores and ETL pipelines to streamline feature engineering and model development
• Build and maintain infrastructure for deploying machine learning models into production environments
• Be a trusted partner for our stakeholders, such as Forecasting, Zestimates, and the ML teams within the AI organization to assist them for data investigation/publication, ML model development and infrastructure support

Requirements:

• Strong proficiency in Python programming
• Hands-on experience with big data ecosystems, architectures and modern data platforms, including AWS cloud data services and Lakehouse/Lakebase platforms
• Proven ability to design, build, and orchestrate batch and real-time data pipelines, processing large-scale datasets from diverse sources
• Practical experience with cutting-edge data engineering technologies such as SparkSQL and Spark Streaming, PostgreSQL, EKS, and Kubernetes
• Familiarity with Agile/DevOps software development processes and tools such as GitLab and CI/CD
• A good understanding of data science concepts, fundamental machine learning algorithms/libraries, and development lifecycle is a must-to-have
• Strong ability in performing root cause analysis to address operational issues and identify opportunities for improvement
• Exceptional communication skills to drive effective customer engagement, collaborate with cross-functional teams, and clearly articulate technical concepts to technical and non-technical stakeholders
• Strong problem-solving skills and the ability to work in a fast-paced, collaborative environment
• 5+ years of experience in Big Data Engineering or Machine Learning Engineering roles
• Hands-on experience with Databricks is a significant advantage
• Familiarity with LLM and Agentic AI technologies and engineering patterns is a big advantage