Invisible Motion uncovered by Eulerian Video Magnification

Some weeks ago I read about a really interesting algorithm proposed by a team of scientists at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory. I found a really good description of how it works on this website. I simply copy and paste here the very good explanation they give about this amazing achievement 🙂

A 30-second video of a newborn baby shows the infant silently snoozing in its crib, his breathing barely perceptible. But when the video is run through an algorithm that can amplify both movement and color, the baby’s face blinks crimson with each tiny heartbeat. The amplification process is called Eulerian Video Magnification, and is the brainchild of a team of scientists at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory.

The team originally developed the program to monitor neonatal babies without making physical contact. But they quickly learned that the algorithm can be applied to other videos to reveal changes imperceptible to the naked eye. Prof. William T. Freeman, a leader on the team, imagines its use in search and rescue, so that rescuers could tell from a distance if someone trapped on a ledge, say, is still breathing. “Once we amplify these small motions, there’s like a whole new world you can look at”, he said.

The system works by homing in on specific pixels in a video over the course of time. Frame-by-frame, the program identifies minute changes in color and then amplifies them up to 100 times, turning, say, a subtle shift toward pink to a bright crimson. The scientists who developed it believe it could also have applications in industries like manufacturing and oil exploration. For example, a factory technician could film a machine to check for small movements in bolts that might indicate an impending breakdown. In one video presented by the scientists, a stationary crane sits on a construction site, so still it could be a photograph. But once run through the program, the crane appears to sway precariously in the wind, perhaps tipping workers off to a potential hazard. It is important to note that the crane does not actually move as much as the video seems to show. It is the process of motion amplification that gives the crane its movement.

The program originally gained attention last summer when the team presented it at the annual computer graphics conference known as Siggraph in Los Angeles. Since then, the M.I.T. team has improved the algorithm to achieve better quality results, with significant improvements in clarity and accuracy. Michael Rubinstein, a doctoral student and co-author on the project, said that after the presentation and subsequent media coverage, the team was inundated with e-mails inquiring about the availability of the program for uses ranging from health care to lie detection in law enforcement. Some people, says Mr. Rubinstein, inquired about how the program might be used in conjunction with Google’s glasses to see changes in a person’s face while gambling. “People wanted to be able to analyze their opponent during a poker game or blackjack and be able to know whether they’re cheating or not, just by the variation in their heart rate”, he said.

The team posted the code online and made it available to anyone who wanted to download it and run the program. But to do so required some technical expertise because the interface was not simple to use. Last week, Quanta Research Cambridge, a Taiwan-based manufacturer of laptop computers that helped finance the project, provided a way for people to upload video clips to their Web site and to see a video that is run through the program. The project is also financed by the National Science Foundation and Royal Dutch Shell, among others.

The team is also working toward making the program as an app for smartphones. “I want people to look around and see what’s out there in this world of tiny motions”, said Mr. Freeman.

Advertisements

Robotique chirurgicale, Robotique de rĂ©habilitation

(extrait du Rapport sur la Recherche et l’Industrie en Robotique en France, 10 Mai 2012)

(auteurs: Michel de Mathelin, Etienne Dombre)

La robotique chirurgicale se dĂ©finit comme la robotique d’assistance au mĂ©decin pour la prĂ©paration et la rĂ©alisation du geste mĂ©dical ou chirurgical. Elle recouvre un large Ă©ventail de systĂšmes robotiques trĂšs diffĂ©rents : robots d’intervention tĂ©lĂ©manipulĂ©s, robots de positionnement d’instruments ou d’ancillaires, systĂšmes de comanipulation, systĂšmes robotisĂ©s d’imagerie ou de radiothĂ©rapie, systĂšmes d’exploration endoluminale, simulateurs avec interfaces Ă  retour d’effort, etc. Un systĂšme robotique mĂ©dical ou chirurgical ne se limite pas au robot lui‐mĂȘme, mais inclut le plus souvent un logiciel de planification pré‐opĂ©ratoire, un logiciel de supervision et de commande du dispositif robotique, ainsi qu’une interface homme machine.

La robotique de rĂ©habilitation se dĂ©finit comme la robotique d’aide aux exercices de rĂ©habilitation de personnes ayant subi un traumatisme. Elle recouvre des dispositifs de sollicitation mĂ©canique de diffĂ©rentes parties du corps, mais aussi de sollicitation de processus cognitifs ou des diffĂ©rents sens de la personne. Le systĂšme robotique ne se rĂ©duit pas au dispositif mĂ©canique et Ă  son actionnement, mais inclut le plus souvent un logiciel de programmation des exercices et un logiciel de supervision enregistrant diffĂ©rents signaux biophysiques. Les systĂšmes de rĂ©habilitation sont également proches des systĂšmes d’entraĂźnement des athlĂštes que l’on peut inclure dans le domaine.

Les robots mĂ©dicaux, chirurgicaux ou de rĂ©habilitation ont diffĂ©rentes particularitĂ©s par rapport à d’autres systĂšmes robotiques :

  •  Ils doivent fonctionner au contact de l’homme en toute sĂ©curitĂ© ;
  •  Ils doivent respecter des normes de fabrication et de conception particuliĂšres ;
  •  Ils doivent montrer un bĂ©nĂ©fice pour le patient ou l’utilisateur.

Ces particularitĂ©s gĂ©nĂšrent des barriĂšres difficiles Ă  franchir en plus de la complexitĂ© inhĂ©rente de la robotique, qui sont : une conception spĂ©cifique du dispositif robotique pour gĂ©rer la sĂ©curitĂ©, des compĂ©tences de fabrication et rĂšglementaires particuliĂšres, une interaction trĂšs Ă©troite avec le milieu mĂ©dical et la mise en oeuvre d’essais complexes.

PhD Thesis

Les principaux laboratoires de recherche en robotique mĂ©dicale sont regroupĂ©s dans un rĂ©seau qui a étĂ© sĂ©lectionnĂ© dans le cadre des Investissements d’avenir Ă  la fois comme Equipement d’excellence (Equipex) : Robotex (http://equipex‐robotex.fr) et comme Laboratoire d’excellence (Labex) : CAMI (http://www.cami‐labex.fr). Il s’agit des laboratoires :

  •  ISIR Ă  Paris ;
  •  LIRMM Ă  Montpellier ;
  •  ICube (anciennement LSIIT) Ă  Strasbourg ;
  •  TIMC Ă  Grenoble.

Il s’agit de laboratoires mixtes entre le CNRS et les universitĂ©s ou Ă©coles du site qui ont la
particularitĂ© d’ĂȘtre implantĂ©s en tout ou en partie au sein de CHU avec des Ă©quipes de recherche mixtes comprenant des chercheurs et des mĂ©decins.
Il faut y ajouter Ă©galement des Ă©quipes de recherche qui travaillent dans le domaine de l’assistance au geste mĂ©dical ou chirurgical sans concevoir de robots, mais en rĂ©alisant des systĂšmes informatiques. On citera :

  •  LaTIM Ă  Brest ;
  •  LTSI Ă  Rennes ;
  •  Equipe Shaman Ă  l’INRIA, etc.

La robotique mĂ©dicale, encore plus que d’autres disciplines de la robotique, est confrontĂ©e Ă  un temps de cycle des projets trĂšs long liĂ© Ă  la nĂ©cessitĂ© impĂ©rieuse de non seulement rĂ©aliser un voire plusieurs prototypes, mais Ă©galement de les tester et valider in vivo. Les investissements sont considĂ©rables et les procĂ©dures d’homologation longues et complexes. Ainsi, peu de robots mĂ©dicaux franchissent le pas du laboratoire pour tenter l’aventure industrielle et encore moins sont des succĂšs commerciaux.

RI-MAN, un robot mĂ©dical, crĂ©Ă© en 2006 au Japon.Cependant, les enjeux en termes d’innovation industrielle sont majeurs Ă©tant donnĂ© l’importance croissante du marchĂ© de la robotique mĂ©dicale, en raison de la demande constante de l’amĂ©lioration de la qualitĂ© de soins, du vieillissement de la population et de la rĂ©duction du nombre de mĂ©decins. L’attente du public (et des mĂ©dias qui survendent parfois ces nouvelles technologies) est trĂšs grande.

La tendance actuelle est aux systĂšmes dĂ©diĂ©s. Ils peuvent ĂȘtre considĂ©rĂ©s comme des outils de chirurgie dotĂ©s d’un actionnement et d’une intelligence car souvent connectĂ©s Ă  un ordinateur. Rentrent dans cette catĂ©gorie tous les robots porte‐aiguille, les robots‐guides pour la chirurgie orthopĂ©dique, les robots pour la chirurgie transluminale ou trocart unique, les cathĂ©ters actifs, les stabilisateurs cardiaques actifs ou encore les robots autonomes de type capsules ingĂ©rables.

How many limbs do you actually perceive?

spidermanA famous movie showed us the advantages of having more limbs than usual. A common person with, for example, six independent arms could be stronger than normal, more dexterous and more efficient in killing annoying spiders. The four additional arms would actually be part of his body, normally connected to his motor nerves in order to be very accurately controlled by his brain. He would also feel pain if one of his additional arms was hurt.

Try to figure out the sensation, I mean the physical sensation, of being able to control each single limb of your body. Focus on all the feelings that you can actually experience with your legs and arms (temperature, pain, sense of touch, shape recognition, EVERYTHING!): it’s amazing. Each part of our body is innervated in a way that determines to which extent we “feel” neuro-biological phenomena. Our nerves get grouped in the spinal cord and the data they collect are transmitted up to the brain. In different areas of the primary motor cortex we can actually find a precise mapping of all the different body regions that are directly responsible for the exchange of sensory and motor information. So, for instance, it is possible to localise the motor cortex area corresponding to the tip of our left thumb.

somatosensory homunculusNow, the fact is that innervation of body regions is not uniform at all. Our lips are clearly more sensitive than our elbows. These differences are respected at the level of the motor cortex. Basing on the amount of cerebral tissue or cortex devoted to a given body region, some smart people proposed a funny representation of primary motor cortex anatomical divisions (picture on the left). In this bizarre cortical homunculus, the size of each body region is proportional to how richly innervated that region is, and not to its actual size. This brain representation develops over time and is different for each individual. What is sure is that our brain “feels” the presence of each single part of our body according to this representation at the level of the primary motor cortex. This leads us to the concept of “body within the brain”:  one’s hand, or toe, or nose exist as soon as it is represented by a specific region of the primary motor cortex.

Now let’s get back to the man with six arms. In his cortical homunculus we will be able to recognize each single arm region. This is due to the fact that, in his case, a cortical reorganization takes place in order to best define its personal “body within the brain” representation, including its peculiarities. Cortical reorganization is somehow continuous during the whole life and corresponds to the redistribution of nerves in those areas that undergo some changes, becoming different than before. This process is not immediate and takes some time. In some cases, as we are going to see, some problems may arise.

Amputees may suffer from the Phantom Limb Syndrome. What’s that? Below here we have a nice definition, found on this website:

Phantom limb syndrome is the perception of sensations, usually including pain, in a limb that has been amputated. Patients with this condition experience the limb as if it were still attached to their body, as the brain continues to receive messages from nerves that originally carried impulses from the missing limb. The exact cause of phantom limb syndrome is unknown. Presumably, the sensations are due to the brain’s attempt to reorganize sensory information following the amputation. The brain must essentially “rewire itself” to adjust to the body change.

phantom limbApproximately, 60 to 80% of amputees suffer from this syndrome. They might feel the sensation of having the missing limb shorter or longer than normal, or positioned in a wrong and painful way, or they may experience tingling, cramping, heat, and cold sensations in the portion of the limb that was removed. Why does this happen? Even if the limb is no longer part of the body, the nerve endings at the amputation site continue to send signals to the brain. At the level of the primary motor cortex, the brain “thinks” the limb is still there! At the same time, slowly, cortical reorganization starts “updating” the distribution of cortex regions.

A more annoying side of Phantom Limb Syndrome is that of Phantom Limb Pain. It is the arising of mild to extreme pain felt in the amputation area. In general, any sensation that the limb could have experienced prior to the amputation may be experienced again and amplify pain conditions. Usually, corresponding to cortical reorganization, phantom limb sensations usually disappear or decrease over time. Some medical treatments usually include heat application, relaxation techniques (massage, electrical nerve stimulation), neurostimulation techniques (such as spinal cord or deep brain stimulation) or surgery (in the case of scar tissues entangling a nerve).

sources: uno, due e tre