Data Engineer II

SelectQuote is an insurance services company that helps consumers find life, home, auto, and senior health insurance. The Data Engineer II develops and manages streaming, batch, and event-based data pipelines, integrates data from multiple sources, and makes it available for reporting and system integration. The role translates business requirements into accurate data solutions, supports complex projects, and guides junior team members.

Responsibilities:

• Manage existing streaming, batching, and event-based data pipeline solutions while looking for opportunities to improve them
• Independently translate business requirements into data solutions while maintaining a high level of accuracy and data integrity
• Independently translates business requirements into technical terms for other team members
• Collaborate with other team members to consolidate the data landscape to reduce the development time and increase consistency among data engineering solutions
• Take ownership of complex projects and ensure their complete and accurate execution with an emphasis on delivering business value
• Recommend changes to project architecture while following architecture guidelines set by senior developers
• Demonstrate quality for junior team members and can provide guidance as needed

Requirements:

• Strong analytical skills
• Objectivity and openness to others views, providing solutions and creative input
• Must be able to adjust when there is a shift in priorities
• Excellent communicator with individuals and groups at all levels
• Detail-oriented, organized, and able to manage user expectations
• Good organizational, time management, and listening skills
• Self-motivated and excited to work with different technologies
• Understanding of ETL concepts and ability to apply those concepts to multiple technologies
• Comfortable working with source control (bitbucket, git, GitHub)
• Bachelor's/Master's degree in Engineering, Computer Science (or equivalent experience) required
• 4+ years of experience engineering data systems required
• 2+ years with Apache Airflow or equivalent training required
• 2+ years of Python development using advanced modules (e.g., Pandas, NumPy) required
• 2+ years using ETL tools required
• 2+ years of troubleshooting stored procedures and other T-SQL required
• Experience working with external REST APIs required
• Experience with AWS ETL services and concepts (Glue, S3, Lambda, boto3)
• Knowledge of various file formats (e.g., JSON, CSV, Parquet, fixed-width)
• Microsoft SQL Server performance tuning
• Prolonged periods of sitting at a desk and working on a computer
• At this time SelectQuote does not hire applicants who solely possess temporary work visas requiring employer sponsorship of any kind including but not limited to OPT, H-1B, L-1, O-1, TN, F-1, J-1, E-3, etc
• Experience with version control (Git CLI) and CI/CD preferred
• Experience working with C# or other statically typed languages is a plus
• Experience with Data Lake development is a plus
• AWS Certified Cloud Practitioner or higher level is preferred
• AWS Certified Data Engineer - Associate is a plus
• Could periodically travel to other office and operational sites