Cloud Data Engineer

Medasource is seeking a Cloud Data Engineer to implement a technology framework for cloud computing, integration, and automation. The role involves designing and developing end-to-end data integration solutions in cloud infrastructure and ensuring the integrity and quality of data services.

Responsibilities:

• Drive scope definition, requirements analysis, data and technical design, pipeline build, product configuration, unit testing, and production deployment
• Design scalable ingestion processes to bring on-prem, API drive, 3rd party, end user generated data sources to integrate in common cloud infrastructure
• Design reusable assets, components, standards, frameworks, and processes to accelerate and facilitate data integration projects
• Develop data integration and transformation jobs using Python, SQL and ETL /ELT tools
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
• Build processes supporting data transformation, data structures, metadata, dependency and workload management
• Design parameter driven orchestration to allow for change data capture and monitoring
• Develop and implement scripts for data process maintenance, monitoring, and performance tuning
• Test and document data processes through data validation and verification procedures
• Collaborate with a cross-functional team to resolve data quality and operational issues
• Ensure delivered solutions meet/perform to technical and functional/non-functional requirements
• Ensure delivered solutions are realized in time frame committed
• Provide technical guidance and mentorship to junior engineers, ensuring best practices in data engineering
• Maintain overall industry knowledge on latest trends, technology, etc

Requirements:

• Must have experience in data transformation and data pipeline development using GUI based tools or programming languages like SQL and Python
• Bachelor's Degree in Computer Science or related field
• 5 years of experience in at least two IT disciplines, including database management, cloud engineering, data engineering and middleware technologies
• Includes 2 years of work experience with cloud platforms, including experience with data integration, performance optimization, and platform administration
• Experience defining, designing, and developing solutions with data integration platforms/tools
• Proven experience building and optimizing data pipelines, and data sets
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
• Hands-on experience working with cloud based modern ELT tools and technologies like Fivetran, HVR, dbt, Airflow etc
• Proficiency in Python and SQL for scripting and building data transformation processes is preferred
• Experience in test automation with a focus on testing integrations, including APIs and data flows between enterprise systems
• Must have experience with DevOps tool chains and processes
• Must have hands-on experience with Snowflake Data Cloud