Data Engineering Developer

Paradigm is a company that makes world-class software and delivers high-quality professional services for the building products industry. They are seeking a Data Engineering Developer who will be responsible for creating and maintaining data pipeline architecture, providing data-driven insights, and addressing data-related issues.

Responsibilities:

• You will be responsible for creating and maintaining optimal data pipeline architecture using Spark, Python, SQL, and other tools
• You will assemble large, complex data sets that meet functional/non-functional business requirements
• You will work with technical and non-technical teams to provide data-driven insights
• You will identify issues related to data, data sources and processing
• You will monitor reporting solutions & make sure that data sets are correct & troubleshoot any issues
• You will be responsible for customer issue tickets - including reviewing existing solutions and looking holistically at our services to identify trends that may need a larger review
• You will document methodologies, variations, and QA processes
• You will intake, identify, document, and share findings and insights to stakeholders, both internal and external
• You may work cross departmentally to be a data SME for other teams

Requirements:

• You have a bachelor's degree (ideally in Comp Sci, Math, Economics, or Statistics)
• You have 5 years' experience in programming or data engineering
• You are very knowledgeable in Python. Knowledge of other JVM languages like Kotlin, Java and Scala, is a plus
• You have advanced working SQL knowledge and experience working with relational databases, query authoring (SQL), as well as working familiarity with a variety of databases
• You have a deep understanding of Apache Spark, especially as part of Azure Databricks
• You are familiar with the Unity Catalog, Spark Declarative Pipelines (aka Delta Live Tables), and Delta Sharing
• You have knowledge of technology related to data transformation and data structures
• You think critically and are not afraid to be wrong (but you love learning from your successes and your mistakes)
• You have excellent requirement gathering and independent problem-solving skills
• You understand AI as a tool to leverage, with a deep understanding of the required output
• You have excellent communication skills (both written & verbal) - including the ability to translate technical concepts to people without technical expertise
• You are organized with strong attention to detail and excellent follow-through
• You are agile and able to adapt to changing demands and manage multiple priorities in a fast-paced environment
• You are self-motivated and resourceful, with strong interpersonal skills; able to work individually as well as with a team