Lead DevOps Engineer (Remote - US)

ICF is a global advisory and technology services provider that combines expertise with technology to help clients solve complex challenges. The Lead DevOps Engineer will own and evolve cloud platforms supporting enterprise applications, operating multi-account AWS environments, automating infrastructure, and strengthening GitOps delivery. The role also involves Kubernetes administration, CI/CD support, security and observability implementation, database operations, and incident response.

Responsibilities:

• Own the architecture and day-to-day operation of multi-account AWS environments across development, QA, staging, and production, including VPC, IAM, EKS, Elastic Load Balancing, CloudFront, Route 53, ACM, S3, and ECR
• Develop and maintain reusable Terraform modules and Terragrunt environment configurations, including remote state, stack dependencies, drift detection, change planning, approvals, imports, and recovery procedures
• Administer Amazon EKS Auto Mode and Kubernetes resources using Helm, Argo CD, and ApplicationSets; manage upgrades, access, capacity, ingress, secrets, and workload reliability
• Support GitOps controllers and Kubernetes infrastructure automation, including AWS Controllers for Kubernetes and Kubernetes Resource Orchestrator, and resolve reconciliation, ownership, and lifecycle failures
• Build and maintain GitHub Actions workflows, AWS OIDC authentication, self-hosted runners, container-image pipelines, environment promotion, deployment approvals, and rollback procedures
• Operate Microsoft SQL Server on Amazon RDS, including parameter and option groups, backups, snapshots, point-in-time recovery, restore testing, Performance Insights, CloudWatch monitoring, and AWS DMS migration workflows
• Implement security and audit controls using IAM, KMS, Secrets Manager, AWS Config, CloudTrail, GuardDuty, Security Hub, Inspector, WAF, security groups, and centralized logging
• Build and maintain observability using Prometheus, Grafana, Amazon CloudWatch, OpenTelemetry and AWS Distro for OpenTelemetry, with actionable dashboards, metrics, logs, traces, alerts, and service-level objectives
• Partner with developers to review and troubleshoot Java and Spring Boot services, including application startup, JVM performance, API behavior, configuration, database connectivity, and container or Kubernetes deployment failures; use logs, metrics, and traces to distinguish application defects from infrastructure issues
• Integrate application and supply-chain security into delivery pipelines, including SAST with SonarQube, DAST, software composition analysis, container-image scanning, dependency checks, and release quality gates
• Lead incident response, root-cause analysis, disaster-recovery exercises, infrastructure upgrades, cost optimization, architecture documentation, operational runbooks, and knowledge transfer

Requirements:

• Bachelor's degree in computer science, information technology, engineering, or a related field, or equivalent professional experience
• 8+ years of DevOps, cloud infrastructure, platform engineering, or site reliability experience, including 5+ years of experience operating production workloads on AWS
• 5+ years of experience with Terraform and infrastructure as code (IaC), including reusable modules, remote state, imports, drift reconciliation, and automated plan and apply workflows
• 4+ years of experience administering production Kubernetes environments, including Amazon EKS, Helm, ingress, RBAC, secrets, upgrades, observability, and GitOps delivery with Argo CD or a comparable platform
• 3+ years of experience building and supporting CI/CD pipelines with GitHub Actions or a comparable platform, plus strong Linux, scripting, AWS networking, IAM, troubleshooting, and incident-response skills
• Must be a US Citizen or Permanent Resident per contract requirements
• Experience taking ownership of and improving an existing production cloud platform, including its architecture, automation, reliability, security, and operational practices
• Hands-on experience with Terragrunt, EKS Auto Mode, AWS Pod Identity, Argo CD ApplicationSets, AWS Controllers for Kubernetes, or Kubernetes Resource Orchestrator
• Experience operating Microsoft SQL Server on Amazon RDS and supporting AWS DMS, backup and restore, disaster recovery, and database performance monitoring
• Experience with CloudFront, Route 53, ACM, Cognito, SQS, SNS, SES, Secrets Manager, S3, ECR, and cross-account AWS access patterns
• Experience implementing observability with Prometheus, Grafana, CloudWatch, OpenTelemetry, centralized logging, alerting, and service-level objectives
• Experience implementing SAST with SonarQube, DAST, software composition analysis, container scanning, SBOMs, and policy-based release gates
• Hands-on experience reviewing and troubleshooting Java and Spring Boot applications, including REST or GraphQL APIs, Maven, JVM diagnostics, database connection pools, and containerized deployments