Associate Data Engineer

BeyondTrust is hiring an Associate Data Engineer to build and enhance its data lake and develop cost-effective cloud data solutions for cybersecurity research. The role focuses on maintaining and optimizing infrastructure, creating data assets and pipelines, and collaborating with engineers to deliver reliable data solutions.

Responsibilities:

• Contribute to data modelling efforts that support identity security use cases and analytical requirements
• Develop and maintain data pipelines that bring together complex information from disparate systems to support security analysis in the datalake
• Optimize data workloads at a software level by improving processing efficiency
• Work with senior engineers to identify areas for process improvement and help implement solutions
• Use monitoring and observability best practices to ensure optimal pipeline performance
• Assist with ML Operations to ensure optimal model efficiency
• Leverage CI/CD best practices to effectively develop and release source code
• Participate in code reviews and collaborate with team members to improve engineering practices

Requirements:

• Programming and technology knowledge in cloud data processing
• Spark experience is needed
• Ability to learn fast and strong desire to develop
• Understanding of software engineering principles, data structures, and distributed data processing concepts
• Experience working with distributed systems and data processing on object stores
• Ability to work autonomously and collaboratively
• Data warehousing for analytics use cases
• Proficient in Python (or another object orientated language)
• Distributed processing framework experience
• existing Databricks knowledge is a plus
• Interest in identity security is preferred
• Spark
• Realtime processing experience
• Some experience with graph data stores