Data Quality Engineer

i4DM is seeking a Data Quality Engineer to support ATLAS, the VA’s enterprise data management and analytics platform. The role designs and automates data validation, testing, profiling, and monitoring to improve data quality, governance, and reliability across cloud-based platforms.

Responsibilities:

• Design and implement data validation and quality assurance processes for cloud-based environments
• Automate data quality checks and develop monitoring solutions to detect data anomalies
• Collaborate with data engineers and analysts to ensure data governance and compliance
• Analyze and profile structured and unstructured datasets for accuracy and consistency
• Build and maintain automated testing frameworks to proactively catch data issues
• Utilize Azure data engineering and ELT tools, including Azure Synapse and Databricks
• Create and execute comprehensive test plans using tools such as JIRA’s X-RAY
• Implement and execute test automation using CoTS tooling like Collibra's Data Quality, Microsoft’s Profisee, or custom scripts in Python or Scala

Requirements:

• 3+ years of experience working in a cloud-based environment
• Experience in analyzing and profiling data
• Proficiency in querying and working with data using SQL and Python
• Experience in building automated testing frameworks for data quality assurance
• Hands-on experience with Azure tools for data engineering and ELT, including Azure Synapse and Databricks
• Strong knowledge of best testing practices and patterns in cloud environments
• Ability to work independently within a mid-sized team and rapidly develop test plans using JIRA's X-RAY
• Proficiency in implementing automated test plans using Collibra's Data Quality, Microsoft Profisee, or script-based testing with Python or Scala
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements
• Bachelor's degree or 4+ years of relevant experience in lieu of a degree
• Experience with Synapse Pipelines, Azure Data Factory, Starburst, Azure Data Lake Service, Azure ML/AI services, and Pyramid Analytics/Power BI for visualization
• Familiarity with JIRA and Confluence for project tracking and documentation
• Ability to integrate testing protocols with monitoring applications such as Dynatrace, Splunk, or Log Analytics
• Bachelor's degree in Computer Science or a related field
• Azure Data Engineer Certification