DevOps Engineer

NEP Group, Inc. is a leader in live production and virtual solutions for content creators. They are seeking a hands-on DevOps Engineer to build and maintain a modern Azure Databricks Lakehouse, focusing on scalability, security, and automation across environments.

Responsibilities:

• Build and maintain Terraform IaC for Azure + Databricks (workspaces, ADLS Gen2, ADF, Key Vault, networking, Unity Catalog, jobs, SQL Warehouses)
• Create a modular, reusable Terraform structure (modules + environment-specific configs) to support dev/prod consistency
• Own remote state + drift detection (state locking, scheduled plans, reproducibility across environments)
• Design and run CI/CD pipelines (Azure DevOps or GitHub Actions) for infrastructure and Databricks deployments using Databricks Asset Bundles (DABs)
• Automate quality gates: linting + unit tests + notebook validation on PRs, with approval gates into production
• Partner with security and platform stakeholders to implement RBAC, SCIM/Entra ID group management, and secret management using Key Vault + secret scopes
• Implement monitoring, alerting, and FinOps controls for job health, data freshness SLAs, and cost anomalies (DBU + Azure budgets/tags)

Requirements:

• Strong Terraform experience (Azure + Databricks providers), including modules, remote state, and multi-environment patterns
• Proven CI/CD ownership (Azure DevOps or GitHub Actions) with automated testing and release controls
• Solid Azure platform knowledge: ADLS Gen2, Key Vault, ADF, Entra ID, VNets/subnets/NSGs, and cost management
• Databricks administration experience: Unity Catalog, cluster policies, SQL Warehouses, secret scopes, and workspace configuration
• Comfort building operational visibility: dashboards, alerts, and usage/cost tracking
• DLT, dbt, Power BI, Collibra, Airflow
• Databricks REST API automation
• Familiarity with data engineering patterns (PySpark/Delta/Medallion)
• Docker/Kubernetes exposure