#940 - Data Engineer

Venon Solutions is seeking a Data Engineer to design and develop scalable data pipelines. The role involves collaborating with data scientists and DevOps teams to ensure secure and efficient data handling and storage solutions.

Responsibilities:

• Designing and developing scalable, efficient, and reliable data pipelines to extract, transform, and load data from various sources to a target system
• Responsible for ingesting data from various sources such as databases, files, APIs, or social media platforms using Python libraries like pandas, NumPy, and requests
• Designing and implementing data storage solutions using relational databases like MySQL or PostgreSQL, NoSQL databases like MongoDB or Cassandra, or big data stores like Hadoop or Spark
• Working closely with data scientists to understand their requirements and implement data pipelines that meet their needs
• Ensuring the security of the data pipeline by implementing access controls, encryption, and authentication mechanisms
• Working closely with DevOps teams to ensure smooth deployment of the data pipeline to production environments on Azure

Requirements:

• B2 English Level or higher
• +5 years of experience in a Data Engineering role
• +4 years of experience with Python
• Experience with Big Data tools such as Hadoop, Spark, Kafka, among others, is expected
• Proven experience with both relational SQL and NoSQL databases, such as PostgreSQL, Cassandra, or MongoDB
• Experience with Azure cloud services is a must
• Good communication skills