Illustration. Brain overheating. Credit: Lion on helium/MIPT Press Office
Researchers from the Moscow Institute of Physics and Technology have developed a model for predicting hand movement trajectories based on cortical activity: Signals are measured directly from a human brain. The predictions rely on linear models. This offloads the processor, since it requires less memory and fewer computations in comparison with neural networks. As a result, the processor can be combined with a sensor and implanted in the cranium. By simplifying the model without degrading the predictions, it becomes possible to respond to the changing brain signals. This technology could drive exoskeletons that would allow patients with impaired mobility to regain movement. The paper was published in Expert Systems with Applications, the leading journal in the field of artificial intelligence.
Damage to the spinal cord prevents signals generated by the brain to control limb motion from reaching the muscles. As a result, the patients can no longer move freely. To restore motion, brain cortex signals are measured, decoded, and transmitted to an exoskeleton. Decoding means interpreting the signals as a prediction of the desired limb motion. To pick up high-quality signals, the sensor needs to be implanted directly in the braincase.
A surgical implantation of a sensor with electrodes onto the motor cortex, the area of the brain responsible for voluntary movements, has already been performed. Such a sensor is powered by a compact battery recharged wirelessly. The device comes with a processing unit, which handles the incoming signals, and a radio transmitter relaying the data to an external receiver. The processor heats up during operation, which becomes problematic since it is in contact with the brain. This puts a constraint on consumed power, which is crucial for decoding the signal.
Adequately measuring brain signals is only one part of the challenge. To use this data to control artificial limbs, movement trajectories need to be reconstructed from the electrocorticogram — a record of the electrical activity of the brain. This is the point of signal decoding. The research team led by Professor Vadim Strijov from MIPT works on models for predicting hand trajectories based on electrocorticograms. Such predictions are necessary to enable exoskeletons that patients with impaired motor function would control by imagining natural motions of their limbs.
Figure 1. In a 2013 study by Yasuhiko Nakanishi et al. published in PLOS One, patients moved three blocks along the sides of a 25-by-25-centimeter square. Signals picked up from the brain of a patient were recorded as electrocorticograms (ECoGs) and matched with the hand movements. Credit: Lion_on_helium/MIPT Press Office
“We turned to linear algebra for predicting limb motion trajectories. The advantage of the linear models over neural networks is that the optimization of model parameters requires much fewer operations. This means they are well-suited for a slow processor and a limited memory,” explains Strijov, the senior author of the paper.
“We solved the problem of building a model that would be simple, robust, and precise,” adds Strijov, who is a chief researcher at MIPT’s Machine Intelligence Laboratory. “By simple I mean there are relatively few parameters. Robustness refers to the ability to retain reasonable prediction quality under minor changes of parameters. Precision means that the predictions adequately approximate natural physical limb motions. To achieve this, we predict motion trajectories as a linear combination of the electrocorticogram feature descriptions.”
Figure 2. For each electrode, one-second signal intervals undergo modifications and are assigned a feature description in amplitude-frequency space — the amplitude is color-coded. The blue curve on the left is a trajectory of limb motion. Each black curve in the middle corresponds to a signal on one electrode. The 3D visualization on the right is the feature description. Credit: A. Motrenko and V. Strijov/Expert Systems with Applications
Each electrode outputs its own signal represented by a frequency and an amplitude. The frequencies are subdivided into bands. The feature description is a history of corticogram signal values for each electrode and each frequency band. This signal history is a time series, a vector in linear space. Each feature is therefore a vector. The prediction of hand motion trajectory is calculated as a linear combination of feature vectors, their weighted sum. To find the optimal weights for the linear model — that is, those resulting in an adequate prediction — a system of linear equations has to be solved.
Figure 3. Each cluster of dots represents an electrode. The color of the dots indicates how close the signal received by the electrode is to the signal of the electrode corresponding to the position of the dot. Yellow means similar, and blue means dissimilar. Such a fractal representation of signal similarity shows that the signals of the top electrodes are the least similar, while the signals of the central electrodes are more alike. Image courtesy of the researchers
Figure 4. Each dot represents an electrode. The color of the dots varies between yellow and blue, indicating the similarity between the signal of the electrode and the limb motion trajectory. The trajectory is given by the three projections on the coordinate axes in the physical experiment. For each axis, there is evidently a set of electrodes suitable for reconstructing the trajectory. This set varies with time. Credit: A. Motrenko and V. Strijov/Expert Systems with ApplicationsHowever, the solution to the system mentioned above is unstable. This is a consequence of the sensors being located close to each other so that neighboring sensors output similar signals. As a result, the slightest change in the signals that are picked up causes a considerable change in the trajectory prediction. Therefore, the problem of feature space dimensionality reduction needs to be solved.
In their future work, the team plans to address the problem of limb trajectory description in the case of a variable brain structure.
Research of this kind conducted at MIPT is meant to contribute to implementing the 17 sustainable development goals outlined in the 2030 Agenda for Sustainable Development (adopted by the United Nations in 2015), as well as the international initiatives for sustainable development and social responsibility.
Strijov explains: “By moving around and getting a response from the environment, humans learn. The structure of the brain changes. New connections form, rendering the model obsolete. We need to propose a model that would adapt to the changes in the brain by changing its own structure. This task is far from simple, but we are working on it.”