Senior Infrastructure Engineering Advisor

Role Overview

• Provide operations, engineering, design, and troubleshooting support for teams on-premises and for Software-as-a-Service (SaaS)-based logging products.
• Work with customers to gather requirements, onboard data, and assist with searches, dashboards, reports, and knowledge objects.
• Devise solutions for technical and business problems as well as adapt new technologies to improve Splunk, Cribl, and cloud operations.
• Support the systems engineering lifecycle activities for large, hybrid Splunk and Cribl deployments, including requirements gathering, design, testing, implementation, operations, and documentation.
• Lead troubleshooting efforts and identify root causes of problems across the enterprise logging environment.
• Work on and lead projects that drive continuous improvement, enhancements of products, services, offerings, and governance.
• Coach and mentor customers and staff that require technical assistance and guidance.
• Partner with Agile Program and Product Management leads to develop, plan, and execute large initiatives.

Requirements

• Master’s degree in Computer Science, Technology Management or related field and 3 years of experience. Will also accept a Bachelor’s degree and 5 years of experience.
• Must have experience with: supporting, operating, and managing large-scale Splunk environments;
• Splunk Cloud and enterprise deployment, configuration, and administration;
• Splunk architecture including Indexing, Search Head Clustering, and data onboarding;
• SPL (Search Processing Language); Query Optimization;
• Data onboarding using Universal Forwarders; HTTP Event Collector (HEC);
• Leading troubleshooting efforts; implementing performance improvements;
• Supporting enterprise-level monitoring solutions; load-balancing data ingestion using TCP/UDP protocols;
• Leveraging native Splunk load-balancing capabilities for optimized performance and scalability; configuring, deploying, and maintaining Cribl Stream and Cribl Edge;
• Cribl log management and data routing; Syslog; Splunk UF/HEC;
• Enterprise-scale telemetry management and data normalization;
• Designing and implementing Cribl Packs for efficient data parsing and advanced data transformation;
• Infrastructure as Code (IaC) automation and configuration tools including AWS CloudFormation, Terraform, Ansible, and Packer;
• AWS security best practices, including IAM policies, encryption methods, and compliance standards;
• Network protocols including TCP/IP, DNS, VLANs, and VPNs; routers, switches, firewalls, load balancers, and wireless access points; firewalls, IDS/IPS, NAC, and network segmentation;
• TCP/UDP Port configuration and performance tuning; Linux operating systems including RHEL, CentOS, and Ubuntu;
• Command-line and shell scripting with Bash, Python, and Perl; configuration management tools including Ansible, Puppet, and Chef;
• CI/CD pipelines and tools including GitLAB CI;
• AWS including EC2, Lambda, ECS, EKS, S3, and Control Tower.

Tech Stack

• Ansible
• AWS
• Chef
• Cloud
• DNS
• EC2
• Firewalls
• Linux
• Packer
• Perl
• Puppet
• Python
• Shell Scripting
• Splunk
• TCP/IP
• Terraform

Benefits

• Flexible work arrangements
• Professional development opportunities