Data Engineer

IMCS Group is seeking a Senior Data Engineer to work remotely while collaborating with Data Scientists and Data Architects. The role involves designing and building scalable data pipelines, operationalizing data solutions on cloud platforms, and ensuring data governance and security.

Responsibilities:

• Data Pipeline Design & Development
• Integration of structured and unstructured data from various sources into data lakes and warehouses
• Build and maintain scalable ETL/ELT pipelines for batch and streaming data using Azure Data Factory, Databricks, Snowflake and Azure SQL Server, control -M
• Collaborate with data scientists, analysts, and platform engineers to enable analytics and ML use cases
• Design, develop, and optimise DBT models to support scalable data transformations
• Cloud Platform Engineering
• They operationalize data solutions on cloud platforms, integrating services like Azure, Snowflake, and third-party technologies
• Manage environments, performance tuning, and configuration for cloud-native data solutions
• Data Modeling & Architecture
• Apply dimensional modeling, star schemas, and data warehousing techniques to support business intelligence and machine learning workflows
• Collaborate with solution architects and analysts to ensure models meet business needs
• Data Governance & Security
• Ensure data integrity, privacy, and compliance through governance practices and secure schema design
• Implement data masking, access controls, and metadata management for sensitive datasets
• Collaboration & Agile Delivery
• Work closely with cross-functional teams including product owners, architects, and business stakeholders to translate requirements into technical solutions
• Participate in Agile ceremonies, sprint planning, and DevOps practices for continuous integration and deployment

Requirements:

• Programming: Python, SQL, Spark
• Cloud Platforms: Azure, Snowflake
• Data Tools: DBT, Erwin Data Modeler, Apache Airflow, API Integrations, ADF
• Governance: Data masking, metadata management, SOX compliance
• Soft Skills: Communication, problem-solving, stakeholder engagement
• 7+ years of data engineering or design experience, designing, developing, and deploying scalable enterprise data analytics solutions from source system through ingestion and reporting
• 5+ years of experience in ML Lifecycle using Azure Kubernetes service, Azure Container Instance service, Azure Data Factory, Azure Monitor, Azure DataBricks building datasets, ML pipelines, experiments, logging, and monitoring. (Including Drifting, Model Adaptation and Data Collection)
• 5+ years of experience in data engineering using Snowflake
• Experience in designing, developing & scaling complex data & feature pipelines feeding ML models and evaluating their performance
• Experience in building and managing streaming and batch inferencing
• Proficiency in SQL and any one other programming language (e.g., R, Python, C++, Minitab, SAS, Matlab, VBA – knowledge of optimization engines such as CPLEX or Gurobi is a plus)
• Strong experience with cloud platforms (AWS, Azure, etc.) and containerization technologies (Docker, Kubernetes)
• Experience with CI/CD tools such as GitHub Actions, GitLab, Jenkins, or similar tools
• Familiarity with security best practices in DevOps and ML Ops
• Experience in developing and maintaining APIs (e.g.: REST)
• Agile/Scrum operating experience using Azure DevOps
• Experience with MS Cloud - ML Azure Databricks, Data Factory, Synapse, among others
• Strong analytical and problem-solving skills and passion for product development
• Strong understanding of Agile methodologies and open to working in agile environments with multiple stakeholders
• Professional attitude and service orientation; team player
• Ability to translate business needs into potential analytics solutions
• Strong work ethic: ability to work at an abstract level and gain consensus
• Ability to build a sense of trust and rapport to create a comfortable and effective workplace