Brooksource is an IT and Engineering services provider that supports Fortune 500 organizations. They are seeking a Security Data Platform Engineer to support and modernize their security data ingestion platforms, focusing on ingestion reliability, platform operations, and infrastructure modernization.
Responsibilities:
- Build, upgrade, and maintain Linux‑based ingestion and analytics infrastructure in AWS, including syslog and Splunk‑adjacent services
- Support modernization efforts such as:
- Upgrading Linux hosts
- Building and deploying syslog services in AWS
- Migrating ingestion pipelines from on‑prem syslog to cloud‑based architectures
- Ensure continuous and reliable ingestion of critical system data for threat detection, analytics, and response
- Develop AWS Lambda functions to pull data from multiple API sources and store it in S3 or a key-value database
- Reduce backlog of data onboardings through repeatable, reliable ingestion patterns
- Support data delivery into Splunk and downstream analytics platforms via existing ingestion and streaming pipelines (e.g., Vector)
- Assist with ingestion‑adjacent Splunk infrastructure work, such as supporting Splunk server upgrades as part of platform lifecycle management and deploying Splunk Edge Processors
- This role is not focused on Splunk content development (dashboards, SPL optimization, detections)
- Enhance ingestion and platform observability using Datadog, including metrics, alerts, and dashboards
- Participate in an on‑call rotation to troubleshoot ingestion and platform incidents
- Create and maintain documentation, diagrams, and runbooks for ingestion pipelines, syslog infrastructure, and platform operations
- Collaborate with internal and external engineering teams to integrate functionality into security data systems
- Work within an Agile development environment to deliver backlog items and meet sprint goals
- Write automated tests to validate code functionality and maintain compatibility across systems
- Contribute to team standards by recommending best practices, tools, and processes for secure data engineering
- Manage priorities effectively in a fast-paced environment to meet deadlines
Requirements:
- Bachelor's degree in Computer Science, Engineering, or equivalent experience
- 3–5 years of experience in platform engineering, data engineering, SRE, or backend engineering roles
- Strong proficiency in Python and experience with AWS Lambda and other serverless services
- Hands-on experience with AWS, including S3, DynamoDB, Lambda, IAM, and EC2‑based services
- Familiarity with Git, Unix/Linux environments, and automated build/test/deploy tools for Python
- Experience working in secure network environments and Agile workflows
- Ability to troubleshoot complex problems, distinguish requirements from preferences, and deliver solutions independently
- Commitment to writing clean, maintainable, and reusable code
- Experience migrating ingestion pipelines from on‑premises environments to AWS
- Familiarity with Vector, Logstash, or similar data streaming tools
- Experience with Datadog or comparable observability platforms
- Working knowledge of Splunk ingestion architectures (forwarders, edge processors) without being a Splunk administrator
- Experience working in security‑sensitive or regulated environments