Invisible Motion uncovered by Eulerian Video Magnification

Some weeks ago I read about a really interesting algorithm proposed by a team of scientists at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory. I found a really good description of how it works on this website. I simply copy and paste here the very good explanation they give about this amazing achievement 🙂

A 30-second video of a newborn baby shows the infant silently snoozing in its crib, his breathing barely perceptible. But when the video is run through an algorithm that can amplify both movement and color, the baby’s face blinks crimson with each tiny heartbeat. The amplification process is called Eulerian Video Magnification, and is the brainchild of a team of scientists at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory.

The team originally developed the program to monitor neonatal babies without making physical contact. But they quickly learned that the algorithm can be applied to other videos to reveal changes imperceptible to the naked eye. Prof. William T. Freeman, a leader on the team, imagines its use in search and rescue, so that rescuers could tell from a distance if someone trapped on a ledge, say, is still breathing. “Once we amplify these small motions, there’s like a whole new world you can look at”, he said.

The system works by homing in on specific pixels in a video over the course of time. Frame-by-frame, the program identifies minute changes in color and then amplifies them up to 100 times, turning, say, a subtle shift toward pink to a bright crimson. The scientists who developed it believe it could also have applications in industries like manufacturing and oil exploration. For example, a factory technician could film a machine to check for small movements in bolts that might indicate an impending breakdown. In one video presented by the scientists, a stationary crane sits on a construction site, so still it could be a photograph. But once run through the program, the crane appears to sway precariously in the wind, perhaps tipping workers off to a potential hazard. It is important to note that the crane does not actually move as much as the video seems to show. It is the process of motion amplification that gives the crane its movement.

The program originally gained attention last summer when the team presented it at the annual computer graphics conference known as Siggraph in Los Angeles. Since then, the M.I.T. team has improved the algorithm to achieve better quality results, with significant improvements in clarity and accuracy. Michael Rubinstein, a doctoral student and co-author on the project, said that after the presentation and subsequent media coverage, the team was inundated with e-mails inquiring about the availability of the program for uses ranging from health care to lie detection in law enforcement. Some people, says Mr. Rubinstein, inquired about how the program might be used in conjunction with Google’s glasses to see changes in a person’s face while gambling. “People wanted to be able to analyze their opponent during a poker game or blackjack and be able to know whether they’re cheating or not, just by the variation in their heart rate”, he said.

The team posted the code online and made it available to anyone who wanted to download it and run the program. But to do so required some technical expertise because the interface was not simple to use. Last week, Quanta Research Cambridge, a Taiwan-based manufacturer of laptop computers that helped finance the project, provided a way for people to upload video clips to their Web site and to see a video that is run through the program. The project is also financed by the National Science Foundation and Royal Dutch Shell, among others.

The team is also working toward making the program as an app for smartphones. “I want people to look around and see what’s out there in this world of tiny motions”, said Mr. Freeman.

wtf Google ?

How SEARCH works – the story 🙂

Because Google should be (together with Wikipedia) one of any PhD student’s best friends !

Robotique chirurgicale, Robotique de réhabilitation

(extrait du Rapport sur la Recherche et l’Industrie en Robotique en France, 10 Mai 2012)

(auteurs: Michel de Mathelin, Etienne Dombre)

La robotique chirurgicale se définit comme la robotique d’assistance au médecin pour la préparation et la réalisation du geste médical ou chirurgical. Elle recouvre un large éventail de systèmes robotiques très différents : robots d’intervention télémanipulés, robots de positionnement d’instruments ou d’ancillaires, systèmes de comanipulation, systèmes robotisés d’imagerie ou de radiothérapie, systèmes d’exploration endoluminale, simulateurs avec interfaces à retour d’effort, etc. Un système robotique médical ou chirurgical ne se limite pas au robot lui‐même, mais inclut le plus souvent un logiciel de planification pré‐opératoire, un logiciel de supervision et de commande du dispositif robotique, ainsi qu’une interface homme machine.

La robotique de réhabilitation se définit comme la robotique d’aide aux exercices de réhabilitation de personnes ayant subi un traumatisme. Elle recouvre des dispositifs de sollicitation mécanique de différentes parties du corps, mais aussi de sollicitation de processus cognitifs ou des différents sens de la personne. Le système robotique ne se réduit pas au dispositif mécanique et à son actionnement, mais inclut le plus souvent un logiciel de programmation des exercices et un logiciel de supervision enregistrant différents signaux biophysiques. Les systèmes de réhabilitation sont également proches des systèmes d’entraînement des athlètes que l’on peut inclure dans le domaine.

Les robots médicaux, chirurgicaux ou de réhabilitation ont différentes particularités par rapport à d’autres systèmes robotiques :

•  Ils doivent fonctionner au contact de l’homme en toute sécurité ;
•  Ils doivent respecter des normes de fabrication et de conception particulières ;
•  Ils doivent montrer un bénéfice pour le patient ou l’utilisateur.

Ces particularités génèrent des barrières difficiles à franchir en plus de la complexité inhérente de la robotique, qui sont : une conception spécifique du dispositif robotique pour gérer la sécurité, des compétences de fabrication et règlementaires particulières, une interaction très étroite avec le milieu médical et la mise en oeuvre d’essais complexes.

Les principaux laboratoires de recherche en robotique médicale sont regroupés dans un réseau qui a été sélectionné dans le cadre des Investissements d’avenir à la fois comme Equipement d’excellence (Equipex) : Robotex (http://equipex‐robotex.fr) et comme Laboratoire d’excellence (Labex) : CAMI (http://www.cami‐labex.fr). Il s’agit des laboratoires :

•  ISIR à Paris ;
•  LIRMM à Montpellier ;
•  ICube (anciennement LSIIT) à Strasbourg ;
•  TIMC à Grenoble.

Il s’agit de laboratoires mixtes entre le CNRS et les universités ou écoles du site qui ont la
particularité d’être implantés en tout ou en partie au sein de CHU avec des équipes de recherche mixtes comprenant des chercheurs et des médecins.
Il faut y ajouter également des équipes de recherche qui travaillent dans le domaine de l’assistance au geste médical ou chirurgical sans concevoir de robots, mais en réalisant des systèmes informatiques. On citera :

•  LaTIM à Brest ;
•  LTSI à Rennes ;
•  Equipe Shaman à l’INRIA, etc.

La robotique médicale, encore plus que d’autres disciplines de la robotique, est confrontée à un temps de cycle des projets très long lié à la nécessité impérieuse de non seulement réaliser un voire plusieurs prototypes, mais également de les tester et valider in vivo. Les investissements sont considérables et les procédures d’homologation longues et complexes. Ainsi, peu de robots médicaux franchissent le pas du laboratoire pour tenter l’aventure industrielle et encore moins sont des succès commerciaux.

Cependant, les enjeux en termes d’innovation industrielle sont majeurs étant donné l’importance croissante du marché de la robotique médicale, en raison de la demande constante de l’amélioration de la qualité de soins, du vieillissement de la population et de la réduction du nombre de médecins. L’attente du public (et des médias qui survendent parfois ces nouvelles technologies) est très grande.

La tendance actuelle est aux systèmes dédiés. Ils peuvent être considérés comme des outils de chirurgie dotés d’un actionnement et d’une intelligence car souvent connectés à un ordinateur. Rentrent dans cette catégorie tous les robots porte‐aiguille, les robots‐guides pour la chirurgie orthopédique, les robots pour la chirurgie transluminale ou trocart unique, les cathéters actifs, les stabilisateurs cardiaques actifs ou encore les robots autonomes de type capsules ingérables.

How many limbs do you actually perceive?

A famous movie showed us the advantages of having more limbs than usual. A common person with, for example, six independent arms could be stronger than normal, more dexterous and more efficient in killing annoying spiders. The four additional arms would actually be part of his body, normally connected to his motor nerves in order to be very accurately controlled by his brain. He would also feel pain if one of his additional arms was hurt.

Try to figure out the sensation, I mean the physical sensation, of being able to control each single limb of your body. Focus on all the feelings that you can actually experience with your legs and arms (temperature, pain, sense of touch, shape recognition, EVERYTHING!): it’s amazing. Each part of our body is innervated in a way that determines to which extent we “feel” neuro-biological phenomena. Our nerves get grouped in the spinal cord and the data they collect are transmitted up to the brain. In different areas of the primary motor cortex we can actually find a precise mapping of all the different body regions that are directly responsible for the exchange of sensory and motor information. So, for instance, it is possible to localise the motor cortex area corresponding to the tip of our left thumb.

Now, the fact is that innervation of body regions is not uniform at all. Our lips are clearly more sensitive than our elbows. These differences are respected at the level of the motor cortex. Basing on the amount of cerebral tissue or cortex devoted to a given body region, some smart people proposed a funny representation of primary motor cortex anatomical divisions (picture on the left). In this bizarre cortical homunculus, the size of each body region is proportional to how richly innervated that region is, and not to its actual size. This brain representation develops over time and is different for each individual. What is sure is that our brain “feels” the presence of each single part of our body according to this representation at the level of the primary motor cortex. This leads us to the concept of “body within the brain”:  one’s hand, or toe, or nose exist as soon as it is represented by a specific region of the primary motor cortex.

Now let’s get back to the man with six arms. In his cortical homunculus we will be able to recognize each single arm region. This is due to the fact that, in his case, a cortical reorganization takes place in order to best define its personal “body within the brain” representation, including its peculiarities. Cortical reorganization is somehow continuous during the whole life and corresponds to the redistribution of nerves in those areas that undergo some changes, becoming different than before. This process is not immediate and takes some time. In some cases, as we are going to see, some problems may arise.

Amputees may suffer from the Phantom Limb Syndrome. What’s that? Below here we have a nice definition, found on this website:

Phantom limb syndrome is the perception of sensations, usually including pain, in a limb that has been amputated. Patients with this condition experience the limb as if it were still attached to their body, as the brain continues to receive messages from nerves that originally carried impulses from the missing limb. The exact cause of phantom limb syndrome is unknown. Presumably, the sensations are due to the brain’s attempt to reorganize sensory information following the amputation. The brain must essentially “rewire itself” to adjust to the body change.

Approximately, 60 to 80% of amputees suffer from this syndrome. They might feel the sensation of having the missing limb shorter or longer than normal, or positioned in a wrong and painful way, or they may experience tingling, cramping, heat, and cold sensations in the portion of the limb that was removed. Why does this happen? Even if the limb is no longer part of the body, the nerve endings at the amputation site continue to send signals to the brain. At the level of the primary motor cortex, the brain “thinks” the limb is still there! At the same time, slowly, cortical reorganization starts “updating” the distribution of cortex regions.

A more annoying side of Phantom Limb Syndrome is that of Phantom Limb Pain. It is the arising of mild to extreme pain felt in the amputation area. In general, any sensation that the limb could have experienced prior to the amputation may be experienced again and amplify pain conditions. Usually, corresponding to cortical reorganization, phantom limb sensations usually disappear or decrease over time. Some medical treatments usually include heat application, relaxation techniques (massage, electrical nerve stimulation), neurostimulation techniques (such as spinal cord or deep brain stimulation) or surgery (in the case of scar tissues entangling a nerve).

sources: uno, due e tre