Senior Azure Databricks Data Engineer / Architect

JPS Tech Solutions is seeking a Senior Azure Databricks Data Engineer / Architect to design and implement scalable data pipelines. The role involves developing robust ETL/ELT workflows and optimizing data architectures using Azure technologies.

Responsibilities:

• Design and implement scalable data pipelines using Azure Databricks and Azure Data Factory
• Develop and maintain robust ETL/ELT workflows using Apache Spark and PySpark DataFrames
• Build and optimize data pipelines for efficient data ingestion, transformation, and processing of large datasets
• Utilize data governance tools to manage data access, security, compliance, and data lifecycle
• Implement data quality frameworks and maintain data lineage across enterprise data platforms
• Design and support modern data architecture using Lakehouse and distributed data processing principles
• Develop high-performance Spark and SQL-based data transformations
• Apply Master Data Management (MDM) concepts to ensure data consistency, integrity, and standardization
• Work within the Azure Databricks ecosystem to deliver scalable, reliable, and enterprise-grade data engineering solutions

Requirements:

• Azure Databricks
• Apache Spark / PySpark
• Azure Data Factory
• SQL
• Data Pipelines and ETL/ELT
• Data Governance and Data Lineage
• Lakehouse Architecture