Lead Data Engineer (AWS, Azure, GCP)

CapTech is an award-winning consulting firm that collaborates with clients to achieve what’s possible through the power of technology. They are seeking a Lead Data Engineer to design, implement, and maintain modern data pipelines while providing technical leadership and collaborating with teams to ensure solutions meet customer needs.

Responsibilities:

• Be trusted advisor to customers with best practices, methodologies, and technologies to implement data engineering solutions
• Design, implement, and maintain modern data pipelines to deliver optimal solutions utilizing appropriate cloud technologies
• Partner with product owners and business SMEs to analyze customer requirements and provide a supportable and sustainable engineered solution
• Provide technical leadership and collaborate within and across teams to ensure that the overall technical solution is aligned with the customer needs
• Stay current with the latest cloud technologies, patterns, and methodologies; share knowledge by clearly articulating results and ideas to key stakeholders

Requirements:

• Experience building/operating highly available distributed systems of data extraction, ingestion, and processing large data sets
• 5+ years of experience delivering data engineering solutions on cloud platform
• 5+ years of experience implementing modern designs using at least one cloud-based solution/platform (AWS, Azure, GCP)
• Advanced level proficiency with at least one ETL / Data Orchestration technology (Azure Data Factory, SSIS, Informatica, Alteryx, Ab Initio, Pentaho, Talend, Matillion)
• Experience cloud-based data warehousing and data lake solutions like Snowflake, Redshift, Databricks
• 5+ years of experience with SQL or NoSQL database (PostgreSQL, MySQL, SQL server, Oracle, Aurora, Presto, BigQuery)
• Expertise with SQL, database design/structure and data structure (star, snowflake schemas, de/normalized designs)
• 5+ years of experience with at least one programming language (Python, Java, R, C / C# / C++, Shell)
• Familiarity with one or more DevOps tools (git, Jenkins, CI/CD, Jira)
• Fundamental understanding of big data, open source, and data streaming concepts
• Ability to think strategically and provide recommendations utilizing traditional and modern architectural components based on business needs
• Experience providing technical leadership and mentoring other engineers in data engineering space
• Cloud certification on any platform a plus