Data Engineer Senior

Progressive Insurance is seeking a senior data engineer to support its Claims IT AI Enablement team. The role focuses on designing, building, and supporting Snowflake-based data pipelines and production data solutions, while optimizing platform performance, ensuring data quality and governance, providing production support, and collaborating with business and technology partners.

Responsibilities:

• You’ll design, build, and support Snowflake-based data pipelines that ingest and transform structured and semi-structured data from cloud sources
• You’ll develop production solutions using Snowpark and Python while applying incremental processing, change data capture, and metadata-driven design patterns
• You’ll also optimize platform performance and cost, implement monitoring and automated recovery capabilities, and ensure data quality, security, and governance
• Additionally, you’ll provide production support through an on-call rotation and collaborate with business and technology partners to resolve issues and continuously improve the data platform

Requirements:

• Bachelor's Degree or higher in an Information Technology discipline or related field of study and minimum of two years of work experience designing, programming, and supporting software programs or applications
• In lieu of degree, minimum of four years related work experience designing, programming, and supporting software programs or applications may be accepted
• Applicants must be authorized to work for any employer in the U.S. without the need or potential need, of current or future sponsorship for employment
• Progressive does not hire candidates with (e.g., F-1 CPT, OPT, or STEM OPT, H-1B, O-1, E-3, TN) statuses for this role
• Experience designing, building, and supporting enterprise data pipelines using Snowflake, Snowpark with Python, Snowpipe, Streams, Tasks, stored procedures, UDFs, and semi-structured VARIANT data
• Experience implementing scalable ingestion patterns for structured and semi-structured data, including JSON processing, cloud-based and event-driven ingestion, schema drift, incremental processing, change data capture, deduplication, and idempotent loading
• Experience designing metadata-driven ingestion and transformation frameworks that support automated data onboarding, schema evolution, reusable pipeline components, configuration management, and governance workflows
• Experience optimizing the performance, reliability, and cost of large-scale data platforms through query profiling, warehouse sizing, clustering, pruning, batching, dynamic tables, monitoring, alerting, testing, and automated recovery
• Experience applying modern data engineering practices, including AWS cloud technologies, CI/CD, Git-based development, infrastructure as code, data governance, and Generative AI or LLM solutions with appropriate governance and human oversight