Security Data Platform Engineer

Brooksource is an IT and Engineering services provider that supports Fortune 500 organizations. They are seeking a Security Data Platform Engineer to support and modernize their security data ingestion platforms, focusing on ingestion reliability, platform operations, and infrastructure modernization.

Responsibilities:

• Build, upgrade, and maintain Linux‑based ingestion and analytics infrastructure in AWS, including syslog and Splunk‑adjacent services
• Support modernization efforts such as:
• Upgrading Linux hosts
• Building and deploying syslog services in AWS
• Migrating ingestion pipelines from on‑prem syslog to cloud‑based architectures
• Ensure continuous and reliable ingestion of critical system data for threat detection, analytics, and response
• Develop AWS Lambda functions to pull data from multiple API sources and store it in S3 or a key-value database
• Reduce backlog of data onboardings through repeatable, reliable ingestion patterns
• Support data delivery into Splunk and downstream analytics platforms via existing ingestion and streaming pipelines (e.g., Vector)
• Assist with ingestion‑adjacent Splunk infrastructure work, such as supporting Splunk server upgrades as part of platform lifecycle management and deploying Splunk Edge Processors
• This role is not focused on Splunk content development (dashboards, SPL optimization, detections)
• Enhance ingestion and platform observability using Datadog, including metrics, alerts, and dashboards
• Participate in an on‑call rotation to troubleshoot ingestion and platform incidents
• Create and maintain documentation, diagrams, and runbooks for ingestion pipelines, syslog infrastructure, and platform operations
• Collaborate with internal and external engineering teams to integrate functionality into security data systems
• Work within an Agile development environment to deliver backlog items and meet sprint goals
• Write automated tests to validate code functionality and maintain compatibility across systems
• Contribute to team standards by recommending best practices, tools, and processes for secure data engineering
• Manage priorities effectively in a fast-paced environment to meet deadlines

Requirements:

• Bachelor's degree in Computer Science, Engineering, or equivalent experience
• 3–5 years of experience in platform engineering, data engineering, SRE, or backend engineering roles
• Strong proficiency in Python and experience with AWS Lambda and other serverless services
• Hands-on experience with AWS, including S3, DynamoDB, Lambda, IAM, and EC2‑based services
• Familiarity with Git, Unix/Linux environments, and automated build/test/deploy tools for Python
• Experience working in secure network environments and Agile workflows
• Ability to troubleshoot complex problems, distinguish requirements from preferences, and deliver solutions independently
• Commitment to writing clean, maintainable, and reusable code
• Experience migrating ingestion pipelines from on‑premises environments to AWS
• Familiarity with Vector, Logstash, or similar data streaming tools
• Experience with Datadog or comparable observability platforms
• Working knowledge of Splunk ingestion architectures (forwarders, edge processors) without being a Splunk administrator
• Experience working in security‑sensitive or regulated environments