Senior ML Infrastructure Engineer
Edison, New Jersey, United States of America
Full Time
1 day ago
H1B Sponsor
Key skills
PythonMLNLPOpenAIPyTorchHugging FaceData LakeAzureTerraformKubernetesDockerVaultAKSHelm
About this role
Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.
Project Overview:
We're building a large-scale document intelligence platform that processes text files up to 5 TB in size, extracts insights using BERT-class NLP models, and surfaces answers to analysts via a low-latency query interface. The platform runs on Azure Kubernetes Service (AKS) with dedicated GPU node pools, uses KEDA for event-driven autoscaling, and integrates with Azure Data Lake Storage Gen2 and Azure OpenAI.
This is a hands-on role that sits at the intersection of platform engineering and applied ML, and requires someone who is equally comfortable debugging a CUDA out-of-memory error and designing a Kubernetes autoscaling policy. As the Senior ML Infrastructure Engineer the resource will own the end-to-end infrastructure layer — from GPU cluster configuration and CUDA runtime management to Kubernetes job orchestration and model serving.
Skill / Technology:
- Level: Kubernetes / AKS
- Expert: Multi-node-pool design, taint/toleration, autoscaler, GPU node pools (NC/ND series)
- Senior: Device plugin, driver compat, resource limits, KEDA
- Senior: Scaled Job, queue triggers, cooldown tuning, CUDA / cuDNN
- Mid–Senior: Runtime config via PyTorch; raw kernel dev not required, PyTorch (GPU inference)
- Senior: Batching, FP16, memory management, profiling, Hugging Face Transformers
- Senior: BERT/DistilBERT/BGE loading, pipeline API, tokenization, Python (production)
- Senior: Async workers, Azure SDK, queue consumers, Azure infrastructure
- Senior: VNet, private endpoints, Key Vault, ADLS, AD, Docker / Helm
- Senior: Multi-stage builds, Helm chart authoring, IaC (Terraform / Bicep)
- Preferred: willingness to learn is acceptable
Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
Candidate Privacy Policy
Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:
- What information we collect during our application and recruitment process and why we collect it;
- How we handle that information; and
- How to access and update that information.
Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.