Security Data Platform Engineer

Brooksource is seeking a Security Data Platform Engineer to support and modernize security data ingestion platforms. The role focuses on ingestion reliability, platform operations, and infrastructure modernization, requiring expertise in secure design, cloud technologies, and automation for data processing pipelines.

Responsibilities:

• Build, upgrade, and maintain Linux‑based ingestion and analytics infrastructure in AWS, including syslog and Splunk‑adjacent services
• Support modernization efforts such as: Upgrading Linux hosts, Building and deploying syslog services in AWS, Migrating ingestion pipelines from on‑prem syslog to cloud‑based architectures
• Ensure continuous and reliable ingestion of critical system data for threat detection, analytics, and response
• Develop AWS Lambda functions to pull data from multiple API sources and store it in S3 or a key-value database
• Reduce backlog of data onboardings through repeatable, reliable ingestion patterns
• Support data delivery into Splunk and downstream analytics platforms via existing ingestion and streaming pipelines (e.g., Vector)
• Assist with ingestion‑adjacent Splunk infrastructure work, such as supporting Splunk server upgrades as part of platform lifecycle management and deploying Splunk Edge Processors
• Enhance ingestion and platform observability using Datadog, including metrics, alerts, and dashboards
• Participate in an on‑call rotation to troubleshoot ingestion and platform incidents
• Create and maintain documentation, diagrams, and runbooks for ingestion pipelines, syslog infrastructure, and platform operations
• Collaborate with internal and external engineering teams to integrate functionality into security data systems
• Work within an Agile development environment to deliver backlog items and meet sprint goals
• Write automated tests to validate code functionality and maintain compatibility across systems
• Contribute to team standards by recommending best practices, tools, and processes for secure data engineering
• Manage priorities effectively in a fast-paced environment to meet deadlines

Requirements:

• Bachelor's degree in Computer Science, Engineering, or equivalent experience
• 3–5 years of experience in platform engineering, data engineering, SRE, or backend engineering roles
• Strong proficiency in Python and experience with AWS Lambda and other serverless services
• Hands-on experience with AWS, including S3, DynamoDB, Lambda, IAM, and EC2‑based services
• Familiarity with Git, Unix/Linux environments, and automated build/test/deploy tools for Python
• Experience working in secure network environments and Agile workflows
• Ability to troubleshoot complex problems, distinguish requirements from preferences, and deliver solutions independently
• Commitment to writing clean, maintainable, and reusable code
• Experience migrating ingestion pipelines from on‑premises environments to AWS
• Familiarity with Vector, Logstash, or similar data streaming tools
• Experience with Datadog or comparable observability platforms
• Working knowledge of Splunk ingestion architectures (forwarders, edge processors) without being a Splunk administrator
• Experience working in security‑sensitive or regulated environments