Senior Data Engineer

SoTalent is seeking a highly skilled Senior Data Engineer to build and optimize scalable data solutions that power enterprise analytics and business intelligence. This role focuses on designing modern data architectures, enabling seamless data integration, and driving cloud-based data transformation initiatives.

Responsibilities:

• Design and develop scalable, high-performance data pipelines using cloud-based platforms
• Collaborate with cross-functional teams to define data requirements and build robust data architectures
• Integrate data across enterprise systems to ensure efficient and reliable data flow
• Implement best practices to maintain high standards of data quality, performance, and reliability
• Support data modernization initiatives through cloud adoption and process optimization
• Build and maintain CI/CD pipelines to automate deployments and streamline workflows
• Utilize cloud storage and orchestration tools to manage data securely and efficiently
• Continuously evaluate emerging technologies and implement industry best practices
• Communicate technical solutions effectively to both technical and business stakeholders

Requirements:

• Bachelor's degree with 8+ years of experience, or Master's degree with 5+ years in Data Engineering or related field
• Advanced SQL for complex querying, debugging, and optimization
• Strong programming experience in Java, Python, and/or Scala
• Hands-on experience with database systems (e.g., MySQL or similar)
• Expertise in ETL/ELT pipeline design and data orchestration
• Strong understanding of data modeling (dimensional modeling, normalization/denormalization)
• Experience with CI/CD processes and tools (e.g., Git-based automation)
• Experience working with enterprise ETL tools and data integration platforms
• Experience with platforms like Databricks and distributed processing frameworks such as PySpark
• Familiarity with cloud-based ETL tools and workflows (e.g., Azure Data Factory or similar)
• Experience with cloud platforms (AWS/Azure) including storage, identity management, and automation services
• Exposure to modern data lake and analytics architectures