Intermediate Data Engineer
United States
Full-time
2 weeks ago
$98,000 - $106,000 USD
No H1B
Key skills
Data pipeline developmentCloud platforms (AWS)Programming languages PythonProgramming languages SQLData lifecycle understandingTechnical documentation writingSoft skillsPythonSQLRShellArtificial IntelligenceMachine LearningETLELTSnowflakeAirflowdbtAWSAzureJenkinsS3Azure DevOpsCI/CD
About this role
Progressive is dedicated to helping employees move forward and live fully in their careers. As an intermediate data engineer on the new IT Data Management Team, you’ll assist in optimizing data and expanding the data pipeline infrastructure while driving faster time-to-insights through automation and enhancing data governance.
Responsibilities:
- Assist in the optimization of our data and expanding the data pipeline infrastructure
- Build data solutions that could be consumed for data science, artificial intelligence, machine learning and other advanced data & analytic solutions
- Help drive faster time-to-insights by automating data orchestration to eliminate manual data preparation bottlenecks
- Enable analysts to extract and analyze data more rapidly decreasing time to insights
- Help eliminate data silos, unlock data, and enhance our data governance by centralizing disparate data sources
- Help position IT data for scalable growth, managing data ingestion and transformation jobs as data complexity and volumes increase
- Research industry trends and Progressive enterprise-wide data/architecture capabilities
- Implement new techniques as necessary
- Provide technical recommendations for advancing our data strategy
Requirements:
- Bachelor's Degree or higher in an Information Technology discipline or related field of study and minimum of one year of work experience designing, programming, and supporting software programs or applications
- In lieu of degree, minimum of two years related work experience designing, programming, and supporting software programs or applications may be accepted
- Experience with building data pipelines for collecting, transforming, and integrating data from various sources. Also, experience calling external API's
- Experience with cloud platforms like AWS (S3) and cloud services like Snowflake with data replication, orchestration, and transformation tools (Prefect, DBT, Airflow, etc. or similar tech stack)
- Experience with programming languages and other technical tools such as Python, R, SQL, SSIS, Power Automate/PowerApps
- Comprehensive understanding of the data lifecycle, from ideation and ETL/ELT processes, through deployment of data products, ensuring efficient data enablement and reporting solutions. Also, experience with CI/CD automation (e.g. Jenkins or Azure DevOps), Unix/Shell scripting, and experience with data warehouses
- Experience writing technical documentation