Senior Data Engineer

Role Overview

• Take ownership of data modelling to solve complex identity security problems.
• Solve complex analytical problems across disparate systems to provide a unified view of security posture in the datalake
• Optimize data workloads at a software level by improving processing efficiency.
• Identify opportunities for engineering process improvement and collaborate with senior resources to execute a plan of action.
• Use monitoring and observability best practices to ensure optimal pipeline performance.
• Assist with ML Operations to ensure optimal model efficiency.
• Leverage CI/CD best practices to effectively develop and release source code.

Requirements

• Strong programming and technology knowledge in cloud data processing.
• Spark experience is needed, existing Databricks knowledge is a plus.
• Interest and aptitude for cybersecurity; interest in identity security is highly preferred.
• Technical understanding of underlying systems and computation minutiae.
• Experience working with distributed systems and data processing on object stores.
• Ability to work autonomously

Tech Stack

• Cloud
• Cyber Security
• Distributed Systems
• Spark