Establish and maintain optimized, scalable data pipelines for authorized end users
Collaborate with cross-functional teams to understand data requirements
Provide technical leadership and mentorship to a team of Data Engineers
Oversee design, development, and optimization of big data solutions and pipelines
Utilize expertise in Databricks with Python, PySpark, and SQL
Design, develop, and implement ITL/ELT pipelines
Conduct in-depth analysis of complex business processes
Monitor and troubleshoot data pipelines for accuracy and performance
Stay updated on industry trends in data engineering

15+ Years of IT experience
8+ years of experience as a Data Architect, Data Engineer or other data professional
Experience with Databricks and ETL tools
Knowledge of Mainframe and VSAM data structures
Hands on with Jenkins pipelines and CI/CD
Expert Relational database concepts and experience across multiple database technologies (e.g., SQL, NoSQL, Oracle, Hadoop, Postgres)
Hands on experience in data engineering desired with extensive knowledge of SQL and PySpark, on Databricks Platform
Hands on experience designing, building, and optimizing data pipelines, architecture for large scale data warehousing projects
Experience with cloud data services such as Redshift, RDS, Snowflake and Databricks
Knowledge of AWS Data services is a big plus
Demonstrated expertise with data sourcing, profiling, data mapping and data analysis
Strong SQL, PL/SQL experience and skilled in creating and optimizing queries and data structures
Knowledge of ETL concepts, preferably Informatica or Syncsort
Experience with one or more SQL-on-Hadoop technology (HIVE, Presto, Spark SQL) and Hadoop ecosystem
Ability to design cloud and on-prem data solutions
Excellent written and verbal communication skills and creative problem-solving skills
Ability to adapt to changing business priorities

Senior Data Engineer

Key skills