Invisible Motion uncovered by Eulerian Video Magnification

Some weeks ago I read about a really interesting algorithm proposed by a team of scientists at the Massachusetts Institute of Technology‚Äôs Computer Science and Artificial Intelligence Laboratory. I found a really good description of how it works on this website. I simply copy and paste here the very good explanation they give about this amazing achievement ūüôā

A 30-second video of a newborn baby shows the infant silently snoozing in its crib, his breathing barely perceptible. But when the video is run through an algorithm that can amplify both movement and color, the baby’s face blinks crimson with each tiny heartbeat. The amplification process is called Eulerian Video Magnification, and is the brainchild of a team of scientists at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory.

The team originally developed the program to monitor neonatal babies without making physical contact. But they quickly learned that the algorithm can be applied to other videos to reveal changes imperceptible to the naked eye. Prof. William T. Freeman, a leader on the team, imagines its use in search and rescue, so that rescuers could tell from a distance if someone trapped on a ledge, say, is still breathing.¬†‚ÄúOnce we amplify these small motions, there‚Äôs like a whole new world you can look at‚ÄĚ, he said.

The system works by homing in on specific pixels in a video over the course of time. Frame-by-frame, the program identifies minute changes in color and then amplifies them up to 100 times, turning, say, a subtle shift toward pink to a bright crimson. The scientists who developed it believe it could also have applications in industries like manufacturing and oil exploration. For example, a factory technician could film a machine to check for small movements in bolts that might indicate an impending breakdown. In one video presented by the scientists, a stationary crane sits on a construction site, so still it could be a photograph. But once run through the program, the crane appears to sway precariously in the wind, perhaps tipping workers off to a potential hazard. It is important to note that the crane does not actually move as much as the video seems to show. It is the process of motion amplification that gives the crane its movement.

The program originally gained attention last summer¬†when the team presented it¬†at the annual computer graphics conference known as Siggraph in Los Angeles.¬†Since then, the M.I.T. team has improved the algorithm to achieve better quality results, with significant improvements in clarity and accuracy.¬†Michael Rubinstein, a doctoral student and co-author on the project, said that after the presentation and subsequent media coverage, the team was inundated with e-mails inquiring about the availability of the program for uses ranging from health care to lie detection in law enforcement. Some people, says Mr. Rubinstein, inquired about how the program might be used in conjunction with Google‚Äôs glasses to see changes in a person‚Äôs face while gambling.¬†‚ÄúPeople wanted to be able to analyze their opponent during a poker game or blackjack and be able to know whether they‚Äôre cheating or not, just by the variation in their heart rate‚ÄĚ, he said.

The team posted the code online and made it available to anyone who wanted to download it and run the program. But to do so required some technical expertise because the interface was not simple to use. Last week, Quanta Research Cambridge, a Taiwan-based manufacturer of laptop computers that helped finance the project, provided a way for people to upload video clips to their Web site and to see a video that is run through the program. The project is also financed by the National Science Foundation and Royal Dutch Shell, among others.

The team is also working toward making the program as an app for smartphones. ‚ÄúI want people to look around and see what‚Äôs out there in this world of tiny motions‚ÄĚ, said Mr. Freeman.

Advertisements

Robotique chirurgicale, Robotique de r√©habilitation

(extrait du¬†Rapport sur la Recherche et l’Industrie en Robotique en France, 10 Mai 2012)

(auteurs: Michel de Mathelin, Etienne Dombre)

La robotique chirurgicale se d√©finit comme la robotique d‚Äôassistance au m√©decin pour la pr√©paration¬†et la r√©alisation du geste m√©dical ou chirurgical. Elle recouvre un large √©ventail de syst√®mes¬†robotiques tr√®s diff√©rents : robots d‚Äôintervention t√©l√©manipul√©s, robots de positionnement¬†d‚Äôinstruments ou d‚Äôancillaires, syst√®mes de comanipulation, syst√®mes robotis√©s d‚Äôimagerie ou de¬†radioth√©rapie, syst√®mes d‚Äôexploration endoluminale, simulateurs avec interfaces √† retour d‚Äôeffort,¬†etc. Un syst√®me robotique m√©dical ou chirurgical ne se limite pas au robot lui‚Äźm√™me, mais inclut le¬†plus souvent un logiciel de planification pr√©‚Äźop√©ratoire, un logiciel de supervision et de commande¬†du dispositif robotique, ainsi qu‚Äôune interface homme machine.

La robotique de r√©habilitation se d√©finit comme la robotique d‚Äôaide aux exercices de r√©habilitation de¬†personnes ayant subi un traumatisme. Elle recouvre des dispositifs de sollicitation m√©canique de¬†diff√©rentes parties du corps, mais aussi de sollicitation de processus cognitifs ou des diff√©rents sens¬†de la personne. Le syst√®me robotique ne se r√©duit pas au dispositif m√©canique et √† son¬†actionnement, mais inclut le plus souvent un logiciel de programmation des exercices et un logiciel¬†de supervision enregistrant diff√©rents signaux biophysiques. Les syst√®mes de r√©habilitation sont¬†√©galement proches des syst√®mes d‚Äôentra√ģnement des athl√®tes que l’on peut inclure dans le domaine.

Les robots médicaux, chirurgicaux ou de réhabilitation ont différentes particularités par rapport à d’autres systèmes robotiques :

  • ¬†Ils doivent fonctionner au contact de l‚Äôhomme en toute s√©curit√© ;
  • ¬†Ils doivent respecter des normes de fabrication et de conception particuli√®res ;
  • ¬†Ils doivent montrer un b√©n√©fice pour le patient ou l‚Äôutilisateur.

Ces particularités génèrent des barrières difficiles à franchir en plus de la complexité inhérente de la robotique, qui sont : une conception spécifique du dispositif robotique pour gérer la sécurité, des compétences de fabrication et règlementaires particulières, une interaction très étroite avec le milieu médical et la mise en oeuvre d’essais complexes.

PhD Thesis

Les principaux laboratoires de recherche en robotique m√©dicale sont regroup√©s dans un r√©seau qui a¬†√©t√© s√©lectionn√© dans le cadre des Investissements d‚Äôavenir √† la fois comme Equipement¬†d‚Äôexcellence (Equipex) : Robotex (http://equipex‚Äźrobotex.fr) et comme Laboratoire d‚Äôexcellence¬†(Labex) : CAMI (http://www.cami‚Äźlabex.fr). Il s‚Äôagit des laboratoires :

  • ¬†ISIR √† Paris ;
  • ¬†LIRMM √† Montpellier ;
  • ¬†ICube (anciennement LSIIT) √† Strasbourg ;
  • ¬†TIMC √† Grenoble.

Il s’agit de laboratoires mixtes entre le CNRS et les universités ou écoles du site qui ont la
particularité d’être implantés en tout ou en partie au sein de CHU avec des équipes de recherche mixtes comprenant des chercheurs et des médecins.
Il faut y ajouter également des équipes de recherche qui travaillent dans le domaine de l’assistance au geste médical ou chirurgical sans concevoir de robots, mais en réalisant des systèmes informatiques. On citera :

  • ¬†LaTIM √† Brest ;
  • ¬†LTSI √† Rennes ;
  • ¬†Equipe Shaman √† l‚ÄôINRIA, etc.

La robotique m√©dicale, encore plus que d‚Äôautres disciplines de la robotique, est confront√©e √† un¬†temps de cycle des projets tr√®s long li√© √† la n√©cessit√© imp√©rieuse de non seulement r√©aliser un voire¬†plusieurs prototypes, mais √©galement de les tester et valider in vivo. Les investissements sont¬†consid√©rables et les proc√©dures d’homologation longues et complexes. Ainsi, peu de robots m√©dicaux¬†franchissent le pas du laboratoire pour tenter l‚Äôaventure industrielle et encore moins sont des succ√®s¬†commerciaux.

RI-MAN, un robot m√©dical, cr√©√© en 2006 au Japon.Cependant, les enjeux en termes d‚Äôinnovation industrielle sont majeurs √©tant donn√© l‚Äôimportance¬†croissante du march√© de la robotique m√©dicale, en raison de la demande constante de l‚Äôam√©lioration¬†de la qualit√© de soins, du vieillissement de la population et de la r√©duction du nombre de m√©decins.¬†L’attente du public (et des m√©dias qui survendent parfois ces nouvelles technologies) est tr√®s grande.

La tendance actuelle est aux syst√®mes d√©di√©s. Ils peuvent √™tre consid√©r√©s comme des outils de¬†chirurgie dot√©s d‚Äôun actionnement et d‚Äôune intelligence car souvent connect√©s √† un ordinateur.¬†Rentrent dans cette cat√©gorie tous les robots porte‚Äźaiguille, les robots‚Äźguides pour la chirurgie¬†orthop√©dique, les robots pour la chirurgie transluminale ou trocart unique, les cath√©ters actifs, les¬†stabilisateurs cardiaques actifs ou encore les robots autonomes de type capsules ing√©rables.

How many limbs do you actually perceive?

spidermanA famous movie showed us the advantages of having more limbs than usual. A common person with, for example, six independent arms could be stronger than normal, more dexterous and more efficient in killing annoying spiders. The four additional arms would actually be part of his body, normally connected to his motor nerves in order to be very accurately controlled by his brain. He would also feel pain if one of his additional arms was hurt.

Try to figure out the sensation, I mean the physical sensation, of being able to control each single limb of your body. Focus on all the feelings that you can actually experience with your legs and arms (temperature, pain, sense of touch, shape recognition, EVERYTHING!): it’s amazing. Each part of our body is innervated in a way that determines to which extent we “feel”¬†neuro-biological phenomena. Our nerves get grouped in the spinal cord and the data they collect are transmitted up to the brain. In different areas of the primary¬†motor cortex we can actually find a precise mapping of all the different body regions that are¬†directly responsible for the exchange of sensory and motor information. So, for instance, it is possible to localise the motor cortex area corresponding to the tip of our left thumb.

somatosensory homunculusNow, the fact is that innervation of body regions is not uniform at all. Our lips are clearly more sensitive than our elbows. These differences are respected at the level of the motor cortex.¬†Basing on¬†the amount of cerebral tissue or cortex devoted to a given body region, some smart people proposed a funny representation of primary motor cortex anatomical divisions (picture on the left). In this bizarre cortical homunculus, the size of each body region is proportional to how richly innervated that region is, and not to its actual size. This brain representation¬†develops over time and is different for each individual. What is sure is that our brain “feels” the presence of each single part of our body according to this representation at the level of the primary motor cortex. This leads us to¬†the concept of “body within the brain”: ¬†one’s hand, or toe, or nose exist as soon as it is represented by a specific region of the primary motor cortex.

Now let’s get back to the man with six arms. In his cortical homunculus we will be able to recognize each single arm region. This is due to the fact that, in his case, a cortical reorganization takes place in order to best define its personal “body within the brain” representation, including its peculiarities. Cortical reorganization is somehow continuous during the whole life and corresponds to the redistribution of nerves in those areas that undergo some changes, becoming different than before. This process is not immediate and takes some time. In some cases, as we are going to see, some problems may arise.

Amputees may suffer from the Phantom Limb Syndrome. What’s that? Below here we have a nice definition, found on this website:

Phantom limb syndrome is the perception of sensations, usually including pain, in a limb that has been amputated. Patients with this condition experience the limb as if it were still attached to their body, as the brain continues to receive messages from nerves that originally carried impulses from the missing limb.¬†The exact cause of phantom limb syndrome is unknown. Presumably, the sensations are due to the brain‚Äôs attempt to reorganize sensory information following the amputation.¬†The brain must essentially ‚Äúrewire itself‚ÄĚ to adjust to the body change.

phantom limbApproximately, 60 to 80% of amputees suffer from this syndrome.¬†They might feel the sensation of having the missing limb shorter or longer than normal, or positioned in a wrong and painful way, or they may¬†experience tingling, cramping, heat, and cold sensations in the portion of the limb that was removed. Why does this happen? Even if the¬†limb is no longer part of the body, the nerve endings at the amputation site continue to send signals to the¬†brain. At the level of the primary motor cortex, the brain “thinks” the limb is still there! At the same time, slowly, cortical reorganization starts “updating” the distribution of cortex regions.

A more annoying side of Phantom Limb Syndrome is that of Phantom Limb Pain. It is the arising of mild to extreme pain felt in the amputation area. In general, any sensation that the limb could have experienced prior to the amputation may be experienced again and amplify pain conditions. Usually, corresponding to cortical reorganization, phantom limb sensations usually disappear or decrease over time. Some medical treatments usually include heat application, relaxation techniques (massage, electrical nerve stimulation), neurostimulation techniques (such as spinal cord or deep brain stimulation) or surgery (in the case of scar tissues entangling a nerve).

sources: uno, due e tre