Embedded AI Engineer, On-Device Models

Deepgram is the leading platform for Voice AI, providing real-time APIs for speech-to-text and text-to-speech. The Embedded AI Engineer will optimize and deploy Deepgram's models on low-power consumer hardware, ensuring efficient performance on resource-constrained devices.

Responsibilities:

• Take Deepgram's Speech and Conversational models and get them running on embedded and low-power consumer hardware — defining the architecture for on-device, real-time inference across a diverse range of processors and accelerators
• Optimize models for constrained targets through quantization, pruning, distillation, operator fusion, and architecture-specific compilation to meet strict latency, memory, power, and thermal budgets
• Write and optimize performance-critical runtime code (C, C++, and/or Rust) for embedded environments, including bare-metal and real-time operating systems such as FreeRTOS and Zephyr
• Integrate with industry-standard edge inference runtimes and vendor NPU/DSP toolchains, mapping model graphs efficiently onto on-device accelerators and CPU/GPU/NPU heterogeneity
• Build the on-device runtime plumbing: model packaging, deployment pipelines, over-the-air update mechanisms, and lightweight telemetry for devices operating with limited or intermittent connectivity
• Establish repeatable benchmarking and validation across target hardware — measuring latency, accuracy, power consumption, memory footprint, and resource utilization — and catch regressions before they ship
• Partner with silicon and device vendors on SDK integration and performance tuning, getting our models to run efficiently on new chipsets and reference platforms
• Collaborate with Research and Engine teams to influence model architectures toward edge-friendly designs from the start, reducing the optimization burden at deployment time

Requirements:

• Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices
• Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments
• Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation
• Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains
• A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance
• Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development
• Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly
• Experience with real-time audio processing on embedded platforms — DSP pipelines, audio codec optimization, wake-word or always-on listening, or streaming inference on microcontrollers and edge SoCs
• Depth in ML optimization techniques — custom quantization schemes, mixed-precision inference, or neural architecture search for edge targets
• Background in hardware evaluation and benchmarking — systematically comparing accelerators, SoCs, or GPUs for specific workload profiles
• Experience shipping AI features in consumer products at scale, and the instinct for what 'production quality' means on a battery-powered device
• Familiarity with model compilation and optimization toolchains and their tradeoffs across hardware targets
• Experience with secure, robust on-device deployment practices — code signing, encrypted model storage, and safe update mechanisms