Sr. Data Engineering Specialist

Leidos is an industry and technology leader serving government and commercial customers with smarter, more efficient digital and mission innovations. They are seeking a Senior Data Engineering Specialist to provide technical leadership in data engineering and support the Global Management Systems Team. The role involves designing and implementing data enrichment pipelines, managing complex data systems, and collaborating within Agile teams to ensure operational excellence.

Responsibilities:

• Provide strategic technical leadership in data engineering, serving as the primary customer-facing expert on DISN network topology data to enable advanced data correlation and enrichment
• Represent the team in high-level technical discussions and decision-making with internal and external stakeholders, influencing solution design and architecture
• Architect, design, develop, document, test, and deploy complex software solutions using Python, JavaScript, Java, and SQL, ensuring scalability and maintainability
• Lead the creation of advanced Kibana visualizations and dashboards, leveraging machine learning capabilities and React to deliver actionable insights
• Oversee the design and implementation of robust data enrichment pipelines, integrating diverse data sources into Confluent (Kafka) and Elastic platforms
• Direct the development and management of multi-site Kafka and Elastic clusters, including deployment, configuration, testing, troubleshooting, and maintenance to support mission-critical operations
• Lead Kafka system integrations between Elasticsearch/Logstash and other enterprise systems
• Guide Agile collaboration, leading scrum teams, mentoring junior engineers, and fostering a culture of continuous improvement and innovation
• Communicate and collaborate effectively with geographically distributed teams to ensure alignment and successful completion of project objectives
• Troubleshoot and resolve complex installation, infrastructure, and system issues; proactively report, monitor, and mitigate technical risks
• Author, review, and maintain comprehensive technical documentation, including DoD requirements, interface documents, and security compliance artifacts
• Ensure all solutions adhere to DoD security standards and guidelines, and provide oversight for platform sustainment and reliability, proactively addressing operational challenges

Requirements:

• Bachelor's degree from an accredited college in a related discipline, or equivalent experience/combined education, with 8-12 years of professional experience; or 6-10 years of professional experience with a related master's degree
• 8+ years of experience in software engineering, data engineering, or business/data analysis, with demonstrated leadership in Agile/Scrum teams
• Extensive hands-on software development experience with Python, Java, SQL, and working knowledge of JavaScript and HTML
• Advanced experience with distributed version control systems such as Git and Bitbucket
• Deep expertise in data analytics and visualization tools, including Kibana, Power BI, Tableau, and the ELK stack (Elasticsearch, Logstash, Kibana)
• Proven experience designing, developing, and optimizing ETL processes and complex data pipelines, including integration with event streaming platforms such as Kafka
• Strong background in data modeling, unification, and analytics to support data-driven projects and advanced enrichment
• Demonstrated ability to architect and implement application and system integrations, including Kafka and Elastic platform integrations
• Solid understanding of networking and internet protocols, with experience supporting network-centric and data-driven environments
• Experience developing and deploying software on UNIX/Linux command line platforms
• Strong facilitation, stakeholder engagement, and communication skills, with the ability to bridge business and technical perspectives and collaborate with customer representatives, domain experts, systems engineers, and architects
• Experience with Agile project management and collaboration tools such as JIRA and Confluence
• Demonstrated leadership in mentoring junior team members and fostering a culture of continuous improvement
• Active Secret DoD Security clearance prior to start date
• Active Security+ Certification (or other applicable DoD 8570 IAT II certification) prior to start date
• Advanced experience with CI/CD techniques, containerized pipelines, and DevOps practices, including architecting and automating end-to-end software delivery processes
• Proven expertise in artificial intelligence and machine learning techniques, with a demonstrated ability to design, build, and integrate AI/ML capabilities into production data platforms
• Deep experience in data integration, storage, and analysis technologies such as Kafka, Elastic, Spark, and NiFi, including architecting scalable, high-performance data solutions
• Extensive experience with Kafka connector integrations, custom connector development, and strong working knowledge of ksqlDB and Kafka Streams for real-time data processing
• Skilled in developing detailed software designs for streaming data applications, particularly using ksqlDB and Kafka Streams
• Demonstrated ability to develop and optimize Kafka system integrations between Elasticsearch/Logstash and other enterprise systems
• Expertise in designing and implementing application deployment pipelines and developing software in containerized environments using Kubernetes and Docker
• Strong familiarity with Kubernetes deployment strategies, Agile methodologies, and related tools for collaborative development
• Experience developing and deploying software in AWS cloud environments, including configuration of cloud infrastructure, networking, and security policies (GovCloud experience preferred)
• Comprehensive experience in full software lifecycle automation (design, development, testing, deployment), including production-grade deployments
• Advanced skills in designing and building automated software testing pipelines using tools such as Ansible, Selenium, JMeter, Junit, and others
• Demonstrated experience developing and deploying software in DoD environments (DISA experience preferred), including building applications that meet DoD security standards, updating code for security scans, and implementing security guidelines (e.g., STIGs)
• Ability to develop DoD requirements, traceability matrices, and detailed project plans/schedules; author software systems engineering documents and interface documents (IDDs/ICDs)
• Extensive experience with Agile methodologies and Atlassian tools, including JIRA and Confluence, for project tracking and collaboration
• Proven ability to work effectively in remote, geographically dispersed teams, demonstrating strong communication and collaboration skills