Staff Security Detection Engineer

Databricks is a leading data and AI company that empowers organizations worldwide to unify and democratize data, analytics, and AI. The Staff Security Detection Engineer will play a critical role in safeguarding the company's products and infrastructure by designing and implementing scalable intrusion detection solutions using advanced machine learning techniques and collaborating with cross-functional teams.

Responsibilities:

• Design and implement advanced detection strategies by deeply understanding and analyzing new or unknown log sources, schemas, and raw data
• Collaborate with cross-functional teams, including product and data engineering teams, to build efficient log ingestion pipelines and support large-scale data analytics
• Engineer and deploy detection solutions on Databricks using Spark, Python, and other cutting-edge technologies with a strong emphasis on clean code, rigorous testing, and comprehensive documentation
• Develop Rule-based and/or ML-based intrusion detection models and integrate them with Databricks' platform, ensuring high accuracy and minimal false positives
• Partner with Incident Response teams to perform threat hunting and to provide detailed logging, alerts, and playbooks, empowering proactive threat detection and response
• Influence the development of long-term technology strategies and roadmaps for detection engineering, ensuring alignment with broader business and security goals
• Represent Databricks at security and engineering conferences, presenting novel detection approaches and thought leadership within the security community

Requirements:

• 10+ years of relevant experience or advanced degree + 7 years of experience, with a focus on security detection engineering
• 6+ years of software engineering experience, with 4+ years specifically in security-related engineering, particularly in detection engineering
• Expertise in securing and operating at least one major cloud environment (AWS, Azure, GCP)
• Strong technical proficiency in key areas such as network security, cloud security, application/log analysis, and endpoint security
• Proven experience in Python, Git/GitHub, and CI/CD automation
• Familiarity with distributed computing environments (e.g., Pyspark), SQL, data analysis tools, and machine learning
• A strong passion for continuous learning and staying updated on evolving attack techniques and defense strategies
• Excellent communication skills, with the ability to collaborate effectively across teams and present complex ideas clearly to stakeholders at all levels
• A leadership mindset with the ability to mentor peers, drive strategic initiatives, and influence the organization's security direction
• terraform knowledge is a plus