The physiological literature on motor control gives a clear constraint for closed-loop BMI systems: the total round-trip time from neural event to peripheral response needs to be short enough that the stimulation-driven feedback loop does not fight the motor system's own predictive control mechanisms. In practice, for upper-limb rehabilitation applications, the relevant window is roughly 30 to 100 milliseconds for the decode-to-stimulate path. Below 30 ms, you are competing with spinal reflex arcs. Above 100 ms, the timing relationship between decoded intent and delivered stimulation becomes perceptually disconnected.
Getting below 50 ms end-to-end on embedded hardware is achievable, but it requires knowing where every millisecond goes. This post breaks down the latency budget stage by stage across the closed-loop BMI pipeline, identifies the stages with fixed versus tunable latency, and describes the tradeoffs in the Ruten middleware architecture that target the tunable ones.
Defining the Latency Budget
The total closed-loop latency is the sum of five distinct delays:
- Acquisition latency: time from neural event at the electrode to the point where the digitized sample is available in the host processor's memory.
- Decoder input latency: time from sample availability to when the decoder has accumulated enough samples to run a decode cycle.
- Decode computation latency: time for the decode algorithm to run on the accumulated buffer and produce an output.
- Command dispatch latency: time from decode output to when the stimulation command reaches the stimulator hardware interface.
- Stimulator response latency: time from command receipt to when the stimulator produces a pulse on the electrode.
Each of these has a different character. Acquisition latency is largely determined by hardware: the analog-to-digital converter's sample rate and the interface transfer protocol. Decoder input latency is a software-architecture choice: how large is the processing window? Decode computation latency depends on algorithm complexity and hardware capability. Command dispatch latency is a firmware and operating system scheduling choice. Stimulator response latency is again hardware-determined.
Acquisition Latency: The Fixed Floor
At a 30,000 Hz sampling rate (a common rate for spike-resolution neural recording), a single sample period is 33 microseconds. The acquisition latency floor is determined by how quickly the ADC output can be transferred to the host processor. For a 64-channel amplifier using USB bulk transfer with a 1 ms USB frame interval, the minimum latency from sampling to host availability is approximately 1 to 2 ms, because USB frames aggregate data over a full frame period before transfer.
For lower-latency applications, SPI-based acquisition with direct DMA into host memory can reduce this to below 500 microseconds at the expense of bus bandwidth and a tighter coupling between the acquisition hardware and the host processor's SPI controller. This is a hardware selection decision that happens before middleware is involved.
The middleware's role in acquisition latency is to not add to it. Ruten's acquisition layer uses a double-buffered DMA scheme where the current acquisition buffer is written by DMA while the previous buffer is available for decoding. The buffer handoff happens through a callback rather than a polling loop, so there is no spinning delay between DMA completion and decoder notification.
Decoder Input Latency: The Dominant Tunable Factor
This is where most of the latency budget is spent, and where most of the design tradeoffs live. The decoder needs a minimum observation window to produce a reliable output. For spike-rate-based motor intent decoders, the minimum window is approximately 50 to 100 ms to get stable firing rate estimates across a neural population. For LFP-based power feature decoders, windows of 100 to 200 ms are common. For surface EMG envelope extraction, the envelope time constant is effectively the window length.
The decoder input latency is therefore at minimum equal to the required observation window length. A 50 ms observation window means the decoder cannot produce an output until it has 50 ms of data. If you want 10 ms output latency from a continuously running decoder that processes overlapping windows, you run the decoder on a 50 ms window that advances by 10 ms each cycle. The output latency is then 10 ms plus computation time, but you are always working with data that is between 10 and 60 ms old.
The Ruten decoder layer supports configurable window size and hop size, with the hop size setting the output rate. Smaller hops reduce output latency but increase CPU load proportionally. For a Cortex-M4 running a 64-channel spike rate decoder with a 50 ms window and a 10 ms hop, the per-cycle computation load is roughly 15 to 20 percent of available cycles at 168 MHz. For an x86 host running the same algorithm, the computation load is negligible.
Decode Computation Latency: Algorithm Choices Matter
The decode algorithm itself contributes latency in two ways: the computation time per cycle, and whether the algorithm introduces additional buffering requirements. Linear decoders such as a Wiener filter or a simple population vector average are fast: they are matrix-vector products over the accumulated feature vector. A 100-channel neural population vector average on a Cortex-M4 completes in under 200 microseconds. More complex decoders, such as a recurrent neural network or a particle filter, can add 1 to 5 ms of compute latency and require floating-point units to stay within budget.
The recurrent network case is worth noting specifically. Recurrent networks that maintain hidden state across decode cycles introduce a form of algorithmic latency because the output at time T depends on inputs going back to time T-K for some history depth K. This can produce sharper decoding than a memoryless linear decoder, but it means the effective latency of the decode output is longer than the window size alone suggests. We recommend accounting for this when building the latency budget.
Command Dispatch and Stimulator Response Latency
For RTOS-based embedded systems, command dispatch latency is determined by task scheduling. If the decode task and the stimulation command task are running at the same priority, the dispatch can be delayed by a full scheduling quantum, which on FreeRTOS with a 1 ms tick is up to 1 ms. For systems where latency is critical, the stimulation command dispatch should be elevated to a high-priority task, or the decode task should directly enqueue the stimulation command in the same execution context without yielding.
Ruten's stimulation command bus uses a priority queue with a configurable high-water mark. Commands above the mark trigger an interrupt-level flush rather than waiting for the next scheduler pass. This keeps dispatch latency below 100 microseconds in practice on Cortex-M4 targets.
Stimulator response latency varies by hardware. Stimulators that accept a command and output the pulse in the same serial transaction have sub-millisecond response times. Stimulators that buffer commands and execute on a trigger signal have variable latency depending on when the trigger fires relative to the command receipt. Both are compatible with the Ruten command bus; the device adapter specifies which model applies and the timing accounting adjusts accordingly.
A Worked Budget at 50 ms Target
For a 64-channel spike-rate decoder on a Cortex-M4, targeting 50 ms end-to-end latency:
- Acquisition (SPI DMA): approximately 0.5 ms
- Decoder input window: 40 ms, hop 10 ms
- Decoder computation (population vector): approximately 0.3 ms
- Command dispatch (priority queue, interrupt flush): approximately 0.1 ms
- Stimulator response (serial, direct pulse): approximately 0.5 ms
- Total: approximately 51 ms at the worst case hop boundary
This budget is tight but achievable on current embedded hardware without FPGA offload. The dominant term is the decoder input window, and that is a deliberate tradeoff: reducing it below 40 ms requires either accepting noisier spike rate estimates or switching to a different feature type with a shorter natural window.
The budget also illustrates why latency optimization without a stage-by-stage breakdown leads to wasted effort. A team that spends two weeks optimizing the dispatch stack to get from 0.5 ms to 0.1 ms has saved 0.4 ms against a 51 ms budget, a less than 1 percent improvement. The same two weeks spent evaluating a shorter-window decoder or a different feature extraction approach could move the budget by 5 to 10 ms. Knowing where the budget goes is the prerequisite to improving it.