Senior Data Engineer

Role Overview

• Collaborate & Lead: Work closely with business product owners, data scientists, analysts, and cross-functional stakeholders to understand the business’ data needs and provide technical solutions.
• Influence business partners to align to the technical solutions and to adhere to technical architecture standards.
• Provide technical guidance to junior engineers, BI developers, and contractors to create efficient and effective data solutions.
• Architecting and Innovate: Strong proficiency in Python, Spark, SQL, PySQL, Pandas, CI/CD methodologies is required.
• Strong data ingestion, data modeling and dimensional modeling skills using medallion lake house architecture.
• Strong BI skills to build reports & dashboards using Power BI and Tableau etc.
• Experience in reporting security like row level, column level, object level and masking etc.
• Experience with SQL and DML to recast data in backend database for data changes, restatements and data processing errors, etc.
• Experience with ML Ops and supporting Data Science workflow pipelines.
• Knowledge of Gen AI frameworks and LLMs to support agentic products
• Optimize and Scale: Build and maintain data pipelines to integrate data from various source systems.
• Optimize data pipelines for performance, reliability and cost-effectiveness.
• Work with enterprise infrastructure and technology teams to implement best practices for performance monitoring, cloud resource management, including scaling, cost control and security.
• Ensure Quality and Governance: Ensure safe custody, transport and storage of data in the data platforms.
• Collaborate with Data Governance Stewards and Business Stakeholders to enforce the business rules, data quality rules and data cataloging activities.
• Ensure data quality, security and compliance for the data products responsible under this role.
• Enhance BI Capabilities: Develop and manage business intelligence solutions for the organization to transform data into insights that can drive business value.

Requirements

• 7+ years of experience if the candidate holds BS degree in Computer Science, Information Systems or relevant streams;
• 5-7 years of experience if the candidate holds MS/PhD degree
• Experience in architecting data solutions, cloud data engineering, end to end data warehouse or lake house implementations, end to end business intelligence implementations
• 7 plus years of experience with data engineering, data warehousing, business intelligence with substantial experience in managing large-scale data projects
• 5 plus years’ experience with data solutions implementations in Cloud platform technologies like Microsoft Azure, AWS etc.
• 4 plus years with business intelligence using technologies like Power BI, Tableau etc.
• 4 plus years of experience with Azure services like Data Factory, Databricks, and Delta Lake will be an added advantage.
• Knowledge or experience in Microsoft D365 Dataverse and reporting in Microsoft Fabric technology

Tech Stack

• AWS
• Azure
• Cloud
• Pandas
• Python
• Spark
• SQL
• Tableau

Benefits

• robust health plans
• market-leading 401(k) program with a company match
• flexible time off benefits
• inclusive fertility/adoption benefits