Deep Neural Networks & the Nature of the Universe

source: MIT Technology Review

In the last couple of years, deep learning techniques have transformed the world of artificial intelligence. One by one, the abilities and techniques that humans once imagined were uniquely our own have begun to fall to the onslaught of ever more powerful machines. Deep neural networks are now better than humans at tasks such as face recognition and object recognition. They’ve mastered the ancient game of Go and thrashed the best human players.Risultati immagini per go game But there is a problem. There is no mathematical reason why networks arranged in layers should be so good at these challenges. Mathematicians are flummoxed. Despite the huge success of deep neural networks, nobody is quite sure how they achieve their success.

Today that changes thanks to the work of Henry Lin at Harvard University and Max Tegmark at MIT. These guys say the reason why mathematicians have been so embarrassed is that the answer depends on the nature of the universe. In other words, the answer lies in the regime of physics rather than mathematics.

First, let’s set up the problem using the example of classifying a megabit grayscale image to determine whether it shows a cat or a dog. Such an image consists of a million pixels that can each take one of 256 grayscale values. So in theory, there can be 2561000000 possible images, and for each one it is necessary to compute whether it shows a cat or dog. And yet neural networks, with merely thousands or millions of parameters, somehow manage this classification task with ease. In the language of mathematics, neural networks work by approximating complex mathematical functions with simpler ones. When it comes to classifying images of cats and dogs, the neural network must implement a function that takes as an input a million grayscale pixels and outputs the probability distribution of what it might represent. The problem is that there are orders of magnitude more mathematical functions than possible networks to approximate them. And yet deep neural networks somehow get the right answer.

Now Lin and Tegmark say they’ve worked out why. The answer is that the universe is governed by a tiny subset of all possible functions. In other words, when the laws of physics are written down mathematically, they can all be described by functions that have a remarkable set of simple propertiesSo deep neural networks don’t have to approximate any possible mathematical function, only a tiny subset of them.

To put this in perspective, consider the order of a polynomial function, which is the size of its highest exponent. So a quadratic equation like y=x2 has order 2, the equation y=x24 has order 24, and so on. Obviously, the number of orders is infinite and yet only a tiny subset of polynomials appear in the laws of physics. “For reasons that are still not fully understood, our universe can be accurately described by polynomial Hamiltonians of low order,” say Lin and Tegmark. Typically, the polynomials that describe laws of physics have orders ranging from 2 to 4.

The laws of physics have other important properties. For example, they are usually symmetrical when it comes to rotation and translation. Rotate a cat or dog through 360 degrees and it looks the same; translate it by 10 meters or 100 meters or a kilometer and it will look the same. That also simplifies the task of approximating the process of cat or dog recognition. These properties mean that neural networks do not need to approximate an infinitude of possible mathematical functions but only a tiny subset of the simplest ones.

There is another property of the universe that neural networks exploit. This is the hierarchy of its structure. “Elementary particles form atoms which in turn form molecules, cells, organisms, planets, solar systems, galaxies, etc.,” say Lin and Tegmark. And complex structures are often formed through a sequence of simpler steps. This is why the structure of neural networks is important too: the layers in these networks can approximate each step in the causal sequence. Lin and Tegmark give the example of the cosmic microwave background radiation, the echo of the Big Bang that permeates the universe. In recent years, various spacecraft have mapped this radiation in ever higher resolution. And of course, physicists have puzzled over why these maps take the form they do.

Risultati immagini per cosmic microwave background radiation

Tegmark and Lin point out that whatever the reason, it is undoubtedly the result of a causal hierarchy. “A set of cosmological parameters (the density of dark matter, etc.) determines the power spectrum of density fluctuations in our universe, which in turn determines the pattern of cosmic microwave background radiation reaching us from our early universe, which gets combined with foreground radio noise from our galaxy to produce the frequency-dependent sky maps that are recorded by a satellite-based telescope,” they say. Each of these causal layers contains progressively more data. There are only a handful of cosmological parameters but the maps and the noise they contain are made up of billions of numbers. The goal of physics is to analyze the big numbers in a way that reveals the smaller ones. And when phenomena have this hierarchical structure, neural networks make the process of analyzing it significantly easier. 

We have shown that the success of deep and cheap learning depends not only on mathematics but also on physics, which favors certain classes of exceptionally simple probability distributions that deep learning is uniquely suited to model,” conclude Lin and Tegmark. That’s interesting and important work with significant implications. Artificial neural networks are famously based on biological ones. So not only do Lin and Tegmark’s ideas explain why deep learning machines work so well, they also explain why human brains can make sense of the universe. Evolution has somehow settled on a brain structure that is ideally suited to teasing apart the complexity of the universe.

This work opens the way for significant progress in artificial intelligence. Now that we finally understand why deep neural networks work so well, mathematicians can get to work exploring the specific mathematical properties that allow them to perform so well. “Strengthening the analytic understanding of deep learning may suggest ways of improving it,” say Lin and Tegmark. Deep learning has taken giant strides in recent years. With this improved understanding, the rate of advancement is bound to accelerate.



karma chameleon (gripper)

source: Festo


The chameleon is able to catch a variety of different insects by putting its tongue over the respective prey and securely enclosing it. The FlexShapeGripper uses this principle to grip the widest range of objects in a form-fitting manner. Using its elastic silicone cap, it can even pick up several objects in a single gripping process and put them down together, without the need for a manual conversion.


The gripper consists of a double-acting cylinder, of which one chamber is filled with compressed air whilst the second one is permanently filled with water. This second chamber is fitted with elastic silicone moulding, which equates to the chameleon’s tongue. The volume of the two chambers is designed so that the deformation of the silicone part is compensated. The piston, which closely separates the two chambers from each other, is fastened with a thin rod on the inside of the silicone cap.


During the gripping procedure, a handling system guides the gripper across the object so that it touches the article with its silicone cap. The top pressurised chamber is then vented. The piston moves upwards by means of a spring support and the water-filled silicone part pulls itself inwards. Simultaneously, the handling system guides the gripper further across the object. In doing so, the silicone cap wraps itself around the object to be gripped, which can be of any shape, resulting in a tight form fit. The elastic silicone allows a precise adaptation to a wide range of different geometries. The high static friction of the material generates a strong holding force.

insideOnce it has been put into operation, the gripper is able to do various tasks. This functional integration is a possible way of how systems and components can in future adapt to various products and scenarios themselves. The project also shows how Festo acquires new findings from nature for its core business of automation. But the aims of the Bionic Learning Network not only include learning from nature. Identifying good ideas and fostering them also plays a major part. The FlexShapeGripper came about through a cooperation with the the Oslo and Akershus University College of Applied Sciences and is an outstanding example for a close collaboration beyond company borders.


10 Things To Never Apologize For Again

source: Jessica Hagy‘s post on Forbes

I’m so sorry, but—” is the introductory phrase of doom. Apologizing when you haven’t made any mistakes makes you look weak and easy to dismiss, not polite. Still want to say sorry? Then just don’t say it in these 10 situations.


1. Don’t apologize for taking up space.

You’re three-dimensional in many powerful ways.


2. Don’t apologize for not being omniscient.

If you really were psychic, you’d be out spending your lottery winnings already.

3. Don’t apologize for manifesting in a human form.


You require food, sleep, and you have regular biological functions. This is not being high-maintenance. This is being alive.

4. Don’t apologize for being intimidatingly talented.


Do you detect a wee bit (or a kilo-ton) of jealousy? Good. You’re doing something more than right.

5. Don’t apologize for not joining the cult du jour.


If you don’t believe in the life-changing magic of the brand synergy matrix (or whatever the slide-show is selling), you’re more aware than you realize.

6. Don’t apologize for being bound by the laws of time and space.


Need to be in three places at once? Actually, no, you don’t.

7. Don’t apologize for not assisting the more-than-able.


Get your own stupid coffee, Chad.

8. Don’t apologize for not being unimpressed by mediocrity.


Work that gets praised gets repeated. Stop clapping for things you don’t ever want to see again.

9. Don’t apologize for trusting your gut.


Don’t walk down the dark creepy alley or into that closed-door meeting with the predator, okay?

10. Don’t apologize for standing up for people you care about. 


Because you’re tired of hearing them apologize for doing everything right.

10 Things Great Leaders Say That Creates Engaged Teams

source: LinkedIn (all rights belong to Gordon Tredgold, the author of the post)

Great leadership is about creating great relationships with your teams and inspiring them to go above and beyond. Here are 10 things that great leaders say to create highly engaged and motivated teams. They cost nothing but the returns can be amazing.


  1. Sorry, my fault. No one is perfect and by owning up to mistakes it builds trust, and it also sets a great example for the rest of the team. When accountability starts at the top, the rest of the team will model it.
  2. What do you need from me to make this a success? This is my favorite approach to leadership as it clearly shows that we are in this together and that their success is one of our concerns, and we are more than happy to contribute to it. It also clarifies whether or not they everything they need to be successful. Once they say I have everything I need, then they have accepted accountability for the outcome which will help increase the probability of success.
  3. I value your contribution. Everyone wants to feel valued and needed as it helps to build confidence and self-esteem. The more confident our teams are the better, as confidence is a key contributor to achieving success. What get’s recognized gets repeated and we start by recognizing contribution, as this will then lead to results.
  4. What did we learn from this that we can use next time? Mistakes are always going to happen, but by asking this question we avoid the blame game and we can look to learn from it and improve for the next attempt. I am a big fan of feed forward rather than feedback. We need to learn how we can avoid mistakes rather than allocate blame.
  5. I have complete faith in you. It pays dividends to let your teams know that you have trust in their abilities as it will help them build trust and self-confidence in themselves. Confidence and self-belief are key contributors to success.
  6. How could we do this better? There is nothing worse than an arrogant know-it-all leader who thinks he’s cornered the market in great ideas. Trust me I know I worked for several. With this one phrase, you dispel that illusion and show that you’re open to input, and that collaboration will help us achieve the best results. You never know where great ideas are going to come from, and it’s never a good idea to close down possible sources of great ideas.
  7. Do you have the capacity to do this now? Too many people struggle to say no to the boss, often committing to the workload that is both unhealthy, and will not lead to success. By asking the question, genuinely and with concern, it will allow people to agree to what is achievable without seriously over committing themselves. It also acts to remind them that we are interested in their health and success. As leaders it’s your job to set people up for success!
  8. Great job! Two of greatest words any employee can hear from their boss. Simple, zero costs and massively impactful. The more you say it, the more you will have to say it as performance will improve. What gets recognized gets repeated and you want to encourage your teams to repeat good performance, and this simple phrase will do that. Zero cost, great return.
  9. Thank You. Politeness costs nothing. A lack of politeness, on the other hand, shows disrespect and a feeling of entitlement, neither of which is going to build trust and loyalty within the team. This simple phrase makes people feel valued, recognized and appreciated, all of which are great motivators.
  10. How are you doing? No one cares how much you know until they know you care‘ is one of my favorite Theodore Roosevelt quotes and the best way to show you care is to ask people how they are doing.

Leadership is often seen as difficult and complex, but by just using these 10 simple phrases it will help you to keep it simple and create highly engaged, empowered and excited teams who will follow you anywhere and will achieve great results.