Tata Consultancy Services is seeking an Azure Data Engineer Lead to work across various Fabric workloads and design data architectures for enterprise-scale analytics. The role involves preparing and enriching data for analysis, ensuring data quality, and collaborating with stakeholders to translate requirements into scalable solutions.
Responsibilities:
- Work across Fabric workloads: COSMSO, Kusto, Data Factory (ETL), Synapse Data Engineering (Spark/SQL), Synapse Data Warehousing (SQL), and OneLake for storage
- Design data loading patterns, Lakehouse architectures, and orchestration processes for enterprise-scale analytics
- Prepare and enrich data for analysis, ensuring data quality and consistency checks
- Implement semantic models for BI and self-service analytics
- Secure and manage analytics assets, enforce governance policies, and monitor compliance and Managed Identity
- Monitor and optimize Data pipelines and workloads for cost efficiency and high availability
- Work closely with architects, analysts, and business teams to translate requirements into scalable solutions
- Enable Power BI integration for visualization and reporting
- Build and manage data pipelines using COSMOS, Azure Data Factory (ADF), Synapse, Fabric pipelines for batch and incremental ingestion
- Automate ETL workflows and implement parameterized pipelines for scalability
- Develop ETL code / pipelines to clean, transform, and convert raw files into optimized formats (Parquet/Delta)
- Create views, stored procedures, and dimensional models for reporting and analytics
- Monitor Dynamic Management Views (DMVs) in Synapse to analyze system health and reduce bottlenecks
- Implement workload management strategies for query performance and resource governance
- Apply role-based authentication, encryption/decryption logic, and compliance controls
- Automate governance tasks to reduce manual intervention
- Build failure monitoring systems integrated with Logic Apps for alerting and proactive issue resolution
Requirements:
- Synapse
- SQL Server
- Python
- Azure Storage
- ADF
- Cosmos
- Kusto
- Digital: PySpark
- Fabric
- Data Engineering
- DataWarehouse
- Work across Fabric workloads: COSMSO, Kusto, Data Factory (ETL), Synapse Data Engineering (Spark/SQL), Synapse Data Warehousing (SQL), and OneLake for storage
- Design data loading patterns, Lakehouse architectures, and orchestration processes for enterprise-scale analytics
- Prepare and enrich data for analysis, ensuring data quality and consistency checks
- Implement semantic models for BI and self-service analytics
- Secure and manage analytics assets, enforce governance policies, and monitor compliance and Managed Identity
- Monitor and optimize Data pipelines and workloads for cost efficiency and high availability
- Work closely with architects, analysts, and business teams to translate requirements into scalable solutions
- Enable Power BI integration for visualization and reporting
- Build and manage data pipelines using COSMOS, Azure Data Factory (ADF), Synapse, Fabric pipelines for batch and incremental ingestion
- Automate ETL workflows and implement parameterized pipelines for scalability
- Develop ETL code / pipelines to clean, transform, and convert raw files into optimized formats (Parquet/Delta)
- Create views, stored procedures, and dimensional models for reporting and analytics
- Monitor Dynamic Management Views (DMVs) in Synapse to analyze system health and reduce bottlenecks
- Implement workload management strategies for query performance and resource governance
- Apply role-based authentication, encryption/decryption logic, and compliance controls
- Automate governance tasks to reduce manual intervention
- Build failure monitoring systems integrated with Logic Apps for alerting and proactive issue resolution
- Strong in Azure Data Services ADF, Azure SQL database, Synapse
- Ability to understand vast amounts of data, identify and fix data issues
- Strong in database and DW concepts
- Excellent technical & analytical skills with strong business acumen
- Strong communication skills
- Must have Microsoft Azure Cloud and services experience
- Generic Managerial Skills, If any
- Azure OpenAI