Senior Data Engineer (Purview Specialist)

Sesheng Company is seeking a seasoned Purview Developer with a deep background in Data Engineering to lead the implementation and optimization of data governance and cataloging strategies. The role involves building robust data pipelines while ensuring adherence to the highest standards of metadata management and data quality within an Azure Cloud environment.

Responsibilities:

• Lead the implementation and optimization of data governance and cataloging strategies
• Build robust data pipelines while ensuring adherence to the highest standards of metadata management and data quality

Requirements:

• Azure Purview Expertise: At least 2+ years of dedicated experience with Microsoft Purview, focusing on data cataloging, governance, and metadata management within Azure Cloud or Hybrid environments
• Data Engineering Core: 8+ years of hands-on software/computer engineering experience, with a heavy focus on data-centric projects
• Advanced Programming: 4+ years of proficiency in Python or Scala. This includes core programming, PySpark, and experience creating/supporting Unit Tests (pytest) and User Defined Functions (UDFs)
• Big Data Architecture: 4+ years building and optimizing large-scale data pipelines and architectures. You should be fluent in tools like ADF, Synapse, Airflow, DBT, or Kafka
• Database Mastery: 4+ years of expert-level T-SQL experience (MS SQL Server, MySQL, etc.)
• Ops Mindset: Proven experience implementing DevOps and DataOps practices to automate and streamline data delivery
• Data Quality: Experience with automated validation frameworks, specifically Great Expectations
• API Design: Familiarity with Swagger/OpenAPI for documenting and testing RESTful APIs
• AI/ML Integration: Hands-on experience with Machine Learning, AI, or Generative AI workflows
• Healthcare Data: Previous experience working with Epic Clarity, Caboodle, or OMOP data models