Senior Data Engineer

Ciklum is a custom product engineering company that supports multinational organizations and scaling startups. They are looking for a Senior Data Engineer to join their team, focusing on full stack data engineering, including ingestion, transformation, and analytics enablement using modern Azure-based data platforms.

Responsibilities:

• Manage OneLake structures and shortcuts for scalable enterprise data access
• Design scalable Lakehouse solutions using Medallion Architecture (Bronze, Silver, Gold)
• Build and optimize Delta Lake tables for analytics, reporting, and AI workloads
• Develop data pipelines using Fabric Data Factory, Spark, and Notebooks
• Create ingestion and transformation workflows for structured and semi-structured data
• Implement orchestration, scheduling, monitoring, and recovery for enterprise pipelines
• Design dimensional models (Star/Snowflake schemas) for BI and semantic layers
• Build curated Gold-layer datasets for analytics and AI consumption
• Support integration with Power BI semantic models and reporting platforms
• Develop batch and incremental pipelines across Azure and external systems
• Orchestrate ETL/ELT workflows using Fabric Pipelines and Azure Data Factory
• Integrate Fabric platforms with APIs, AI services, and enterprise applications
• Support MCP integration, AI workflows, and rapid prototyping initiatives
• Collaborate on ReactJS-based apps, dashboards, and AI-driven user experiences
• Automate workflows using Azure Functions, Logic Apps, Git, CI/CD, and Azure DevOps
• Develop and optimize PySpark notebooks for transformation, cleansing, and enrichment
• Build efficient SQL queries, views, and stored procedures in Fabric Warehouse / Azure SQL
• Implement optimization techniques including partitioning, caching, and query tuning
• Monitor pipeline performance, troubleshoot failures, and improve system reliability
• Implement logging, alerting, and operational best practices
• Utilize AI-assisted development tools such as GitHub Copilot and modern AI coding assistants
• Rapidly prototype and deliver scalable engineering solutions with minimal guidance
• Implement RBAC and secure data access across Fabric workspaces and Azure environments
• Apply data quality validations and governance best practices
• Support metadata management and lineage using Microsoft Purview
• Collaborate with Data Architects, Analysts, BI Developers, Product Teams, and Business Stakeholders
• Translate business requirements into scalable data and application solutions
• Participate in Agile delivery processes, code reviews, and pull request workflows

Requirements:

• 6 years in Data Engineering
• Hands-on exposure to Microsoft Fabric (preferred) or strong Azure Data Engineering background with willingness to learn Fabric
• Microsoft Fabric (Data Factory, OneLake, Synapse Data Engineering – basics)
• Azure Data Services: ADLS Gen2, Azure SQL, Blob Storage
• Strong SQL (joins, aggregations, performance tuning basics)
• Python (PySpark) for data transformation
• Azure Data Factory / Fabric Pipelines
• Azure Functions / Logic Apps
• Event Hubs / streaming concepts
• Microsoft Purview (basic exposure)
• Cosmos DB