I should say that it is artificial neural networks that seem to need sleep, or a rest. But isn’t one point of an artificial network that that of it not needing sleep, or rest?
For those of you who don’t know, artificial neural networks are networks built by engineers in the computing space to mimic the brain’s cells and therefore hope to get better computing, or different, computing outcomes. Therefore, it comes as a real surprise that, apparently, rest improves their performance. It comes as a real surprise because these are not biological entities, there are numerous reason we need rest as human beings, as biological beings. One is that there is a slow build-up of toxic material as we function. Another is that there is also constant genetic damage that needs to be repaired.
So why do these networks need rest? First let’s understand what happens. These networks are designed to mimic neuronal functions — so far so good — and they have become really good at some things such as computational speed. But there is something called catastrophic forgetting — no, not like when we stand in the supermarket and can’t for the life remember what we wanted. This is when these networks learn sequentially, one task after another, new information can then overwrite old information and it is gone, “forgotten”.
