Data Engineer

NTT DATA is a global business and technology services company. The Data Engineer will design, develop, implement, and optimize scalable Azure data pipelines, data warehouses, Lakehouses, and enterprise data solutions, collaborating with architects, developers, engineers, analysts, and business stakeholders.

Responsibilities:

• Design, develop, and maintain enterprise data solutions using Microsoft Azure
• Build scalable data ingestion, transformation, and processing solutions
• Develop and optimize data pipelines using Azure Data Factory, Azure Synapse Analytics, and/or Microsoft Fabric
• Design solutions that integrate data from databases, APIs, files, SaaS platforms, cloud applications, and enterprise systems
• Develop reusable data engineering frameworks and components
• Implement batch and near-real-time data processing solutions
• Monitor, troubleshoot, and optimize Azure data workloads
• Support modernization and migration of legacy data platforms to Azure
• Design and develop data solutions using Azure Synapse Analytics and/or Microsoft Fabric
• Build enterprise data warehouses and Lakehouse solutions
• Develop data pipelines and orchestration workflows
• Work with Synapse Pipelines, Fabric Data Factory, and Azure Data Factory
• Develop notebooks using Python, PySpark, and Spark SQL
• Build and optimize data transformation processes
• Work with Synapse SQL, Fabric Data Warehouse, Fabric Lakehouse, and related technologies
• Support migration and modernization initiatives between traditional Azure architectures, Azure Synapse, and Microsoft Fabric
• Evaluate appropriate technologies based on performance, scalability, cost, security, and business requirements
• Design and develop robust ETL and ELT pipelines
• Ingest structured, semi-structured, and unstructured data from multiple enterprise sources
• Integrate data from databases, APIs, files, SaaS applications, and cloud platforms
• Develop reusable ingestion and transformation frameworks
• Implement incremental data loads and Change Data Capture (CDC) strategies
• Develop data validation, reconciliation, and error-handling processes
• Troubleshoot pipeline failures and data integration issues
• Optimize pipelines for performance, scalability, reliability, and cost
• Design and implement modern Data Lake, Lakehouse, and Data Warehouse architectures
• Work with Azure Data Lake Storage Gen2 and Microsoft OneLake
• Implement Bronze, Silver, and Gold data layers
• Design dimensional data models using Star and Snowflake schemas
• Develop fact and dimension tables
• Implement Slowly Changing Dimensions (SCD)
• Work with Delta Lake and related technologies
• Implement partitioning and performance optimization strategies
• Support enterprise data warehouse modernization and migration initiatives
• Work with Microsoft Azure technologies including:
• Azure Data Factory
• Azure Synapse Analytics
• Azure Data Lake Storage Gen2
• Azure SQL Database
• Azure SQL Managed Instance
• Azure Functions
• Azure Key Vault
• Azure Event Hubs
• Azure Storage
• Microsoft Entra ID
• Microsoft Fabric
• Develop integrations between Azure services and enterprise applications
• Implement secure and scalable cloud data architectures
• Support cloud migration and modernization initiatives
• Monitor Azure resources and optimize platform performance and cost
• Apply cloud architecture and engineering best practices across data solutions
• Develop solutions using Fabric Lakehouse and Data Warehouse
• Build pipelines using Fabric Data Factory
• Develop Dataflows Gen2
• Work with OneLake
• Develop Fabric Notebooks using PySpark, Python, and Spark SQL
• Implement Delta Lake-based data architectures
• Support Direct Lake and Power BI integration
• Manage Fabric workspaces and deployment processes
• Monitor and optimize Fabric capacity utilization
• Develop scalable data transformation

Requirements:

• The ideal candidate will have strong hands-on experience with Microsoft Azure data services, Azure Synapse Analytics and/or Microsoft Fabric, data engineering, data integration, data transformation, and enterprise data platforms
• Candidates should have significant experience working with Azure Synapse Analytics or Microsoft Fabric, with experience across both platforms considered highly desirable
• 5+ years of experinece in Azure Data Engineering
• 5+ years of experinece in Design, develop, and maintain enterprise data solutions using Microsoft Azure
• 5+ years of experinece in Build scalable data ingestion, transformation, and processing solutions
• 5+ years of experinece in Develop and optimize data pipelines using Azure Data Factory, Azure Synapse Analytics, and/or Microsoft Fabric
• 5+ years of experinece in Design solutions that integrate data from databases, APIs, files, SaaS platforms, cloud applications, and enterprise systems
• 5+ years of experinece in Develop reusable data engineering frameworks and components
• 5+ years of experinece in Monitor, troubleshoot, and optimize Azure data workloads
• 5+ years of experinece in Support modernization and migration of legacy data platforms to Azure
• 5+ years of experinece in Azure Synapse Analytics & Microsoft Fabric
• 5+ years of experinece in Develop notebooks using Python, PySpark, and Spark SQL
• 5+ years of experinece in Design and develop robust ETL and ELT pipelines
• 5+ years of experinece in Design and implement modern Data Lake, Lakehouse, and Data Warehouse architectures
• 5+ years of experinece in Work with Azure Data Lake Storage Gen2 and Microsoft OneLake
• 5+ years of experinece in Develop integrations between Azure services and enterprise applications
• 5+ years of experinece in Develop solutions using Fabric Lakehouse and Data Warehouse