Data Engineer

METONEUM is a company empowering businesses with intelligent digital ecosystems and tailored IT solutions. They are seeking a Data Engineer to design and develop robust data pipelines, manage database schemas, and support data scientists with their data needs.

Responsibilities:

• Design and develop robust data pipelines to extract, transform, and load (ETL) data from various sources into data storage solutions
• Combine data from diverse sources, including databases, APIs, and external data feeds, into a unified system
• Cleanse and preprocess data to ensure accuracy and consistency before it is used for analysis or reporting
• Develop and manage database schemas, including relational and non-relational databases
• Monitor and optimize database performance to ensure efficient data retrieval and storage
• Create and maintain data warehouses to consolidate large volumes of data from different sources for analysis and reporting
• Create and implement data models that represent business processes and data relationships effectively
• Work with big data technologies such as Hadoop, Spark, and Kafka to handle large-scale data processing and real-time data streaming
• Implement security measures to protect sensitive data and comply with relevant regulations and standards
• Control access to data systems and ensure that only authorized users can access or manipulate data
• Collaborate with data scientists to understand their data needs and ensure that data pipelines and infrastructure support their analytical requirements
• Continuously monitor data systems for performance issues, errors, or failures
• Maintain comprehensive documentation for data pipelines, database schemas, and data models
• Create reports on data processing activities, system performance, and data quality for stakeholders

Requirements:

• 1+ years of experience
• Combine data from diverse sources, including databases, APIs, and external data feeds, into a unified system
• Cleanse and preprocess data to ensure accuracy and consistency before it is used for analysis or reporting
• Develop and manage database schemas, including relational and non-relational databases
• Monitor and optimize database performance to ensure efficient data retrieval and storage
• Create and maintain data warehouses to consolidate large volumes of data from different sources for analysis and reporting
• Create and implement data models that represent business processes and data relationships effectively
• Work with big data technologies such as Hadoop, Spark, and Kafka to handle large-scale data processing and real-time data streaming
• Implement security measures to protect sensitive data and comply with relevant regulations and standards
• Control access to data systems and ensure that only authorized users can access or manipulate data
• Collaborate with data scientists to understand their data needs and ensure that data pipelines and infrastructure support their analytical requirements
• Continuously monitor data systems for performance issues, errors, or failures
• Maintain comprehensive documentation for data pipelines, database schemas, and data models
• Create reports on data processing activities, system performance, and data quality for stakeholders