Data Engineer II - OIT

Emory University is a leading research university that fosters excellence and attracts world-class talent to innovate today and prepare leaders for the future. The Data Engineer II is responsible for developing and evolving the Unified Data Platform, collaborating with researchers and cross-functional teams to design scalable data solutions that support research and analytics.

Responsibilities:

• Collaborate as a core member of cross-functional teams including Business Analysts, Project Managers, Data Analysts, and Architects to deliver high-quality solutions within scope and timeline
• Work directly with researchers and stakeholders to gather, analyze, and translate requirements into technical solutions
• Design, develop, and maintain scalable data pipelines, ETL/ELT processes, and data integration workflows across multiple systems
• Build and optimize data solutions that integrate data from disparate sources, ensuring data quality, integrity, and consistency
• Evaluate emerging technologies and develop proof-of-concepts to support innovation and continuous improvement
• Apply biomedical informatics standards, methodologies, and principles to research data solutions
• Ensure adherence to HIPAA and institutional data governance policies and standards
• Develop and maintain metadata, data standards, and data governance processes for complex datasets
• Create and maintain clear technical documentation, including data pipelines, workflows, and system designs
• Communicate technical concepts effectively to both technical and non-technical stakeholders
• Partner with data and system architects and demonstrate an understanding of data modeling concepts and best practices
• Manage workload effectively and provide timely updates on task progress and deliverables
• Works as a positive team member of a project that may consist of Business Analysts, Project Managers, Information Architects, Data Analysts, and/or Database Administrators to deliver quality applications and components within scope, on time, and within budget
• Manages workload effectively and report status of tasks in a timely manner
• Works directly with researchers to document, analyze, and translate their needs into technical designs and informatics solutions
• Participates in the evaluation of emerging technologies and develops proof-of-concepts
• Contributes to technical teams
• Follows standard operational procedures and HIPAA regulations
• Develops strategies for managing complex data sets through maintaining data standards and metadata
• Applies biomedical informatics technical standards, methodologies, and principles to research-specific program needs, objectives, and outcomes
• Develops complex reports, data pipelines, and ETL processes from disparate systems and ensures their accuracy
• Gathers user requirements and creates technical documentation
• Performs other related duties as required

Requirements:

• A bachelor's degree in a related field and three years of related experience, OR an equivalent combination of education, training, and experience
• Strong experience designing and maintaining data pipelines and ETL/ELT processes
• Proficiency in Python, SQL, and Apache Spark (PySpark)
• Experience with modern data platforms such as Azure Fabric (preferred), Azure Synapse Serverless, or Databricks
• Ability to work with large, complex, and distributed datasets
• Solid understanding of data modeling concepts and best practices
• Familiarity with data governance, metadata management, and data quality frameworks
• Knowledge of HIPAA and healthcare data compliance standards
• Experience with CI/CD practices and proficiency with version control systems like Git
• Familiarity with data pipeline orchestration and monitoring tools Like Airflow, Azure Data Factory or similar frameworks
• Understanding of biomedical informatics methodologies is a plus
• Strong analytical, problem-solving, and critical-thinking abilities
• Excellent communication and collaboration skills across diverse teams
• Ability to adapt quickly to new technologies and evolving data environments
• Exposure to FHIR, OMOP, or similar healthcare data models (nice-to-have)