Sr Data Engineer

BeyondTrust is a global identity security leader dedicated to creating a safer world through its cybersecurity SaaS portfolio. As a Senior Data Engineer, you will enhance the datalakehouse, optimize data workloads, and develop solutions to generate actionable insights from vast data sets.

Responsibilities:

• Optimize data workloads at a software level by improving processing efficiency
• Develop new data processing routes to remove redundancy or reduce transformation overhead
• Perform complex data transformation in order to accomplish niche business use cases
• Develop new frameworks for data development to supercharge our delivery velocity
• Monitor and maintain existing data workflows
• Use observability best practices to ensure pipeline performance
• Perform complex transformations on both real time and batch data assets
• Create new ML/Engineering solutions to tackle existing issues in the cybersecurity space
• Leverage CI/CD best practices to effectively develop and release source code

Requirements:

• Strong programming and technology knowledge in cloud data processing
• Previous experience working in matured data lakes
• Strong data modelling skills for analytical workloads
• Spark (or equivalent parallel processing framework) experience is needed
• Ability to work autonomously is a MUST
• Required: Data warehousing for analytics use cases
• Required: Highly proficient in Python (or another object orientated language)
• Required: Distributed processing framework experience
• Existing Databricks knowledge is a plus
• Interest and aptitude for cybersecurity; interest in identity security is highly preferred
• Technical understanding of underlying systems and computation minutiae
• Experience working with distributed systems and data processing on object stores
• Highly Preferred: Spark
• Highly Preferred: Realtime processing experience
• Ideal: Some experience with graph data stores