Edge ML, Embedded Engineer

Role Overview

• Assess target edge hardware against the requirements of an on-device inference loop: evaluate processor architecture, available memory, OS and runtime environment, and whether candidate edge runtimes (such as IoT Greengrass or equivalent) can be supported.
• Evaluate candidate edge inference frameworks for CPU-only SLM deployment — including TensorFlow Lite, ONNX Runtime, llama.cpp, and equivalents — assessing quantization approaches, inference latency, and memory footprint against feasibility targets confirmed during discovery.
• Assess real-time data ingestion feasibility from operational subsystem interfaces, evaluating candidate patterns for consuming concurrent data streams within the memory and compute constraints of the target hardware.
• Design and evaluate local data store options for the on-device SLM context, including storage formats, retrieval latency, and update mechanisms appropriate for the edge environment.
• Build a constrained feasibility demonstrator on laptop or workstation hardware using simulated data feeds.
• Implement a small number of scoped interaction flows in the demonstrator, integrating the voice interface pipeline with the SLM inference and local data retrieval components as agreed through the engagement scope.
• Collaborate with the AI/ML Architect on SLM selection, domain restriction approach, and inference pipeline design — providing hardware and runtime constraint inputs that shape what is architecturally feasible.
• Collaborate with the AWS Solutions Architect on the edge-to-cloud data channel, identifying what can realistically be buffered and transmitted from a constrained edge device under variable connectivity conditions.
• Document hardware assessment findings, framework evaluations, and architectural trade-offs as Architecture Decision Records (ADRs) with explicit rationale.
• Clearly flag where recommendations are conditional on hardware or interface specifications not yet confirmed.
• Communicate technical constraints and feasibility findings clearly to both technical architects and non-technical client stakeholders throughout the engagement.

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent professional experience in embedded systems or edge computing.
• 5+ years of hands-on experience in embedded systems engineering, edge computing, or on-device machine learning, with demonstrated work on constrained hardware environments.
• Expert-level proficiency with at least one edge ML inference framework: TensorFlow Lite, ONNX Runtime, llama.cpp, or equivalent.
• Experience optimizing and quantizing models for CPU-only inference is required.
• Strong understanding of memory management, real-time data stream handling, and concurrent processing in resource-constrained environments.
• Experience with C++, Rust, or Python with tight memory management is strongly preferred.
• Experience with embedded Linux or equivalent OS environments, including ARM-based processors, limited RAM, and environments without GPU availability.
• Familiarity with real-time data ingestion from hardware interfaces or industrial systems — including serial protocols, message bus architectures, or event-driven pipelines at the edge.
• AWS familiarity preferred, specifically IoT Greengrass as a candidate edge runtime and IoT Core for device-to-cloud connectivity.
• Hands-on implementation experience is not required but direct familiarity strengthens the candidate’s ability to evaluate candidate architectures.
• Experience with voice-to-text or text-to-speech pipelines in offline or low-connectivity environments is a plus.
• Comfortable operating in a Phase 0 discovery and feasibility mode — producing assessment findings, ADRs, and a constrained demonstrator rather than production-ready software.
• Strong written communication skills with the ability to document hardware constraint findings, framework evaluations, and architectural trade-offs in formats usable by both technical architects and client stakeholders.
• Experience working in consulting or client-facing project environments is preferred.

Tech Stack

• AWS
• C++
• Cloud
• IoT
• Linux
• Python
• Rust
• Tensorflow

Benefits

• Remote work from home
• Flexible work hours