Senior Data Engineer(W2 role)

CO is seeking a Senior Data Engineer to design and implement scalable data pipelines leveraging Databricks and PySpark within enterprise data ecosystems. The role involves integrating data engineering solutions with the Client’s Data Quality framework, optimizing data workflows, and translating technical insights into business narratives.

Responsibilities:

• Design and implement scalable data pipelines leveraging Databricks and PySpark within enterprise data ecosystems
• Integrate data engineering solutions with the Client’s Data Quality (DQ) framework and standards
• Develop, optimize, and monitor data workflows for cost efficiency and high performance
• Analyze intake/enhancement requests to build and maintain Design Documents and Source-to-Target Mappings (STTMs)
• Translate technical insights into concise business narratives and communicate effectively with stakeholders

Requirements:

• Hands on experience with Databricks, PySpark, and enterprise cloud data platforms
• Demonstrated ability to develop and execute unit test cases, SIT test plans, and manage testing within ADO or similar tools
• Experience executing testing cycles, managing defect lifecycle, and reporting progress
• Ability to support UAT cycles including environment preparation and defect triage
• Experience developing reports and dashboards using Power BI
• Data Engineer: 10 years
• Data Bricks: 10 years
• PySpark: 10 years
• ADO: 8 years