Signal Theory of Intelligence

14 Time analysis in the brain – transition to RNNs

14.1.1 Why Recurrence is Necessary: Static vs. Time-Dependent Signal Processing

First-generation artificial neural networks (ANNs) process exclusively static input signals. An input vector is propagated through the network, and the result depends solely on this single state. Such models are suitable for tasks such as pattern recognition in images or the classification of individual data points, but do not take temporal changes into account.

In real life, however, signals are dynamic. They change over time, and it is precisely these changes that often form the decisive patterns. Speech, music, movements, visual scenes, emotional reactions and internal bodily states are time-dependent processes. The brain must therefore recognise not only static patterns, but also sequences, transitions, rhythms and trends.

To process such temporal patterns, AI research developed recurrent neural networks (RNNs). RNNs possess feedback loops that incorporate past states into current processing. This enables them to represent, learn and recognise temporal components in signals. They are thus functionally closer to biological systems, which also use recurrent loops to integrate temporal information.

The brain fundamentally operates in a recurrent manner. Every perception, every decision and every movement arises from the integration of current signals with short-term and long-term echoes of past activity. This recurrence is not a technical addition, but a fundamental principle of biological intelligence.

14.1.2 Recurrence in the brain: two systems, two time constants

The brain possesses two recurrent signalling systems that operate in parallel and have completely different time constants. This duality is functionally necessary because different learning and processing mechanisms require different echo durations.

14.1.3 Basal ganglia – short-term echoes in the millisecond range 

The basal ganglia generate extremely short-lived recurrent signals, typically in the range of 10–50 milliseconds. Among other things, these echoes determine the temporal resolution of the visual system. Historically, the perception threshold lay at around 18–20 frames per second (≈ 50 ms echo). Modern generations exhibit finer temporal resolution, which is reflected in the acceptance of higher frame rates (60–100 Hz). The nervous system has functionally adapted to this within a few generations. The basal ganglia are thus suited to rapid gain modulation, action selection and the sequencing of motor micro-processes – but not to slow learning processes.

14.1.4 Papez circuits – long-term echoes lasting minutes to hours 

The recurrent limbic loops of the Papez circuit generate significantly longer-lasting patterns of activity. These long-term echoes stabilise emotional, motivational and episodic content and provide the temporal basis for learning processes that take seconds to minutes. The pontocerebellum relies on such long echoes, as LTD/LTP mechanisms do not operate in the millisecond range. The short echoes of the basal ganglia would be unsuitable for this purpose.

Functional consequence 

The existence of two recurrent systems with different time constants is not an anatomical curiosity, but a functional necessity. Fast and slow feedback loops fulfil different roles in signal processing and learning. A single recurrent system could not fulfil these requirements simultaneously. These two systems (basal ganglia/Papez circuits) are connected in parallel in the brain in such a way that their output reaches the recurrent networks together. The signals from the Papez circuits are available for learning via LTP/LTD, whilst those from the basal ganglia are used for rapid recognition.

14.1.5 Evolution of Recurrence: From Single Neurons to the First Differential Circuits

Early vertebrate brains did not consist of networks, but of individual neurons that transmitted sensory signals directly to muscles or to autonomic systems. This architecture allowed for simple reflexes, but not for pattern recognition over time.

The decisive step came with the development of inhibitory neurons. For the first time, these enabled two signal pathways of different lengths to be routed to a common output neuron, with one of the pathways acting as an inhibitor. The difference in propagation time between the two pathways meant that time-invariant signals cancelled each other out, whilst temporal changes – i.e. movements – generated a difference signal.

This circuit was the prototype of the basal ganglia system: an early recurrent structure capable of detecting movements before complex neural networks existed.

The first signal pathway arose from the need for motor control. The second, inhibitory signal pathway developed because averaging systems of the autonomic nervous system require as much input as possible – including motor signals. Through the inhibitory backprojection to the signal sources in the cortex, an early difference signal representing movements was already generated there.

As evolution progressed, the inhibitory interneurons became projection neurons that inhibited the motor signals present in the motor output nucleus (nucleus ruber). This gave rise to the first explicit differential representation of motor function, which was later extended to digitalised signals.

14.1.6 Recurrent neural networks – basic principles

Following the development of biological convolutional neural networks (CNNs) in the primary brain system, the need arose to analyse not only current patterns but also their temporal sequence. This was achieved through modified CNNs, which, in addition to the current input, could also receive time-delayed signals as echoes.

This gave rise to a new class of neural networks: Recurrent Neural Networks (RNNs).

• They operate using feedback loops in which signals are delayed by one time step and reused as input.
• In this way, RNNs can not only recognise spatial patterns but also process temporal sequences.
• Biologically, this corresponds to the brain’s ability to analyse and predict movements, speech or rhythmic patterns.

RNNs thus mark the transition from the purely spatial pattern recognition of CNNs to temporal pattern analysis. It was only on this basis that the next stage of evolution could later emerge – Transformer networks, which enable complex divergence and parallel processing. The brain and AI use the same algorithms, but differ in their technical implementation.

14.2 The technical implementation of RNNs in the vertebrate brain

In the chapter on the emergence of CNNs in the vertebrate brain, I already pointed out that in the vertebrate brain, particularly in primates and Homo sapiens, there are a total of four signal copies arising from the cortical signals:

·        The projection to the ipsilateral nucleus olivaris and from there to the climbing fibre projection of the contralateral cerebellum (climbing fibre projection)

·        The projection via the pontine nuclei to the mossy fibre system of the contralateral cerebellum

·        The projection into the basal ganglia system (matrix and striosomes)

·        The projection into the Papez circuits of the limbic system.

For the technical implementation of biological CNNs, the signal copying in the Papez circuits is crucial.

14.2.1 The Papez circuits as rotational memory

The Papez circuits of the limbic system can be understood as biological rotational memories for neural signals. They originally arose from the need to retain the scent of prey in memory for a prolonged period – an evolutionary advantage, as olfactory signals dissipated too quickly in the waters of the primordial ocean.

In the hippocampus, these scent signals were transformed into a higher-frequency signal sequence. The mossy fibres caused a time delay by stretching the signals along the axon. The Schaffer collaterals tapped into these mossy fibres and converged on a common output neuron, which integrated the distributed action potentials and generated a higher-frequency excitation.

This excitation was fed into the associated Papez circuit, where it could rotate for an extended period.

Signal flow in the Papez circuit

1.     Hippocampus

2.     → Fornix

3.     → Corpora mamillaria (mammillary bodies)

4.     → Mamillothalamic tract

5.     → Anterior thalamic nucleus

6.     → Cingulate gyrus

7.     → Cingulate cortex / Cingulate gyrus

8.     → Entorhinal cortex / parahippocampal gyrus

9.     → back to the hippocampus

Signals were fed into the Papez circuits via three possible entry routes.

Via the cingulate cortex → Cingulum → Hippocampus

·         Association cortices (prefrontal, parietal, temporal) project into the cingulate gyrus.

·         From there, the signals travel via the cingulum to the entorhinal cortex.

·         The entorhinal cortex is the main gateway to the hippocampus.

Directly via the entorhinal cortex

·         The entorhinal cortex receives massive projections from almost all association areas.

·         It projects directly into the hippocampus (perforant path).

Via the amygdala / septum (extended limbic circuit)

·        More recent models (MacLean, Yakovlev) extend the Papez circuit to include the amygdala, septum and orbitofrontal cortex.

From the Papez circuits, the rotating signals can be relayed to the cerebellum, using the following indirect projection pathways:

Via the hypothalamus → brainstem → cerebellum

The Papez circuit contains the mammillary bodies, which are part of the hypothalamus. From the hypothalamus, there are projections to:

·         Nucleus reticularis tegmenti pontis (NRTP)

·         Pontine nuclei

These project extensively into the cerebellum (particularly the vermis and hemispheres).

Via the anterior thalamus → cortex → pons → cerebellum

The Papez circuit sends signals via the anterior thalamic nucleus to the cingulate cortex. From there:

1.     Cingulate cortex → motor / prefrontal areas

2.     → Pontine nuclei

3.     → cerebellum (corticopontocerebellar tract)

This is the most important functional pathway.

Almost all signals can find their way into the Papez circuits. The cingulate gyrus consists of millions of axons, each of which is assigned to a rotating signal.

With each cycle, the cerebellum receives a copy of the signal, which is fed into the mossy fibre system. This copy of the signal remains available even after the original signal has ceased. In this way, the Papez circuits preserve past signals for the present by capturing them in rotational loops and allowing them to rotate there for an extended period as signal echoes.

The existing neural network in the cerebellum developed into a Recurrent Neural Network (RNN) through the feeding in of these signal echoes.

14.2.2 Note on the hippocampus

In the context described here, the hippocampus serves merely to generate rotational signals, which remain active as echoes even after the original signal has ended. There is evidence of learning processes that suggest the existence of neural networks in the hippocampus, but their exact mode of operation is not yet fully understood. The author of this monograph therefore deliberately refrains from a detailed description and limits himself to noting that the hippocampus, as a source of time-delayed signals, plays an important role in feedback within RNNs.

14.2.3 Addendum: Why not all Purkinje cells learn the same signal

A recurring question for the new RNN is: Why do not all Purkinje cells learn the same signal, even though the rotating Papez signals are permanently available to all Purkinje cells?

The answer lies in the topology of real neural networks. These do not use point-to-point connections, but rather a bus topology with lateral inhibition in the afferent signal pathway ( ). In RNNs, lateral inhibition is also extended to the Papez circuits. In such a structure, the signals are not continuously made available to all recipients simultaneously; instead, the rotation is repeatedly interrupted by the network architecture. This results in temporal and spatial differences in the signal flow, which prevent all Purkinje cells from learning identical patterns.

These interruptions are functionally necessary: they lay the foundation for diversity in learning and enable individual Purkinje cells to specialise in different patterns.

14.2.4 Bus topology and lateral inhibition in biological neural networks

Artificial neural networks (ANNs) use a layer topology:

• All neurons in a layer are arranged in a single layer.
• The output neurons are located in the next layer.
• Every neuron in the lower layer is connected to every neuron in the upper layer.
• Every input is passed on to all neurons in the subsequent layer – even if it has already been recognised in a node.
• This redundant forwarding significantly increases energy consumption.

The brain, on the other hand, operates using a bus topology, which is also common in computers:

• All input lines of a layer run in parallel in a data bus beneath the neurons.
• Each neuron (node) is connected to every input line.
• All output lines also run parallel to one another beneath the neurons.
• The neurons lie on an imaginary horizontal line and form a chain of nodes.
• In this bus topology, signals propagate (for example) from left to right without changing direction.
• In each node, there is an afferent signal pathway that carries the input signals, and an efferent signal pathway that carries the signals onwards after processing.
• Via inhibitory interneurons, each neuron can intervene in the efferent signal pathway: if a complex signal (a pattern) is recognised, its input components are removed from the efferent signal pathway and are no longer available to subsequent nodes.

The bus topology of biological networks thus enables economical signal processing:

• Recognised patterns are immediately removed from the data stream.
• Subsequent neurons do not need to relearn or process the same patterns.
• Energy consumption and redundancy are drastically reduced.

The difference between layer topology and bus topology is therefore fundamental: whilst artificial networks rely on maximum parallelism, the brain operates with a sequential, economical signal distribution, which is further safeguarded by lateral inhibition.

Figure: Deep neural network with bus topology (schematic diagram)

14.2.5 Signal-specific, relative inhibition in the efferent signal pathway

Inhibition in the efferent signal pathway is also based on Hebb’s principle and is signal-specific:

• It occurs when the recognising node and its active input lines are active simultaneously.
• In this case, inhibitory interneurons are excited by the recognising node.
• These interneurons form synaptic, inhibitory connections to the active input lines (axons).
• Such axo-axonal connections are known in the nervous system and enable the targeted suppression of those signal components that form the recognised complex signal for the subsequent nodes.

Consequences for signal processing:

• Only the elementary signals of the recognised complex signal are inhibited in the efferent signal pathway.
• Subsequent neurons no longer receive these input signals at all, or only in a greatly attenuated form.
• This creates the necessary break in symmetry within the network: neurons specialise in different signal components rather than learning redundant patterns.
• The method leads to a significant saving in energy and neural activity.

Significance for artificial neural networks:

• The adoption of signal-specific, relative inhibition together with bus topology could bring enormous advantages to ANNs:
• Reduced energy consumption through targeted signal suppression.
• Shorter computation times by avoiding redundant processing.
• Faster learning, as only relevant signal components are passed on.

14.2.6 Language acquisition via RNNs in the brain

The ability to acquire language relies heavily on the temporal processing of signal sequences, which is made possible by RNNs in the brain. Whilst CNNs recognise spatial patterns, RNNs are capable of storing, comparing and linking temporally sequential signals.

Basic principles:

• Language consists of sequences of sounds that unfold over time.
• Thanks to their feedback loops, RNNs can take into account not only the current sound but also the preceding sounds.
• This gives rise to the ability to recognise and reproduce syllables, words and sentences as temporal patterns.

Biological implementation:

• Papez circuits and other feedback loops provide time-delayed copies of signals, which are fed into the cerebellum.
• There, they are combined with the current inputs, enabling sequences of sounds to be analysed.
• Lateral inhibition prevents all neurons from learning the same sequences; instead, individual neurons specialise in different sound sequences.
• Hebbian learning and synaptic plasticity (LTP/LTD) ensure that frequent sound patterns are stored stably.

Evolutionary significance:

• Originally, this ability served to recognise rhythms and sound sequences in the environment.
• With the development of the human brain, it became the basis for language acquisition.
• Children use these RNN structures to extract and stabilise the statistically most frequent patterns from the sound sequences they hear – a process that enables the formation of vocabulary.

14.2.7 Supplement: Language comprehension in lower vertebrates

However, lower vertebrates also develop a certain degree of language comprehension. Dogs and cats clearly respond to their names – they ‘understand’ them in the sense of a learned sequence of sounds. Parrots can even parrot entire sentences. This proves that they can analyse language, and thus possess RNN structures, even if they do not grasp the meaning of these sentences.

This makes it clear that the ability to process temporal sequences of sounds is deeply rooted in evolution. In humans, it has become the foundation of language acquisition and complex speech production, whilst in other species it is used in a reduced form for communication and interaction.

14.2.8 Feedback of recognised complex signals into the Papez circuits

When the original signals and their temporal echoes are processed in the RNN, the output reaches (among other areas) the cortex and, descending, the cerebellum. But they also reach the limbic system with its Papez circuits, where they are converted back into signal echoes and passed on to the cerebellum. This results in deep-layered CNNs in the human brain that can process phonemes, words, sentences and language itself.

14.2.9 The role of the cingulate gyrus in biological RNNs

In the human cingulate gyrus, millions of axons run through which the signals circulate in the associated Papez circuits. They are the physical evidence for the existence of vast neural networks with recurrent signal feedforward. Yet even lower vertebrates must already possess precursors of this architecture, as a certain degree of language comprehension has been demonstrated in numerous animal species.

14.3 CNNs without cerebellar cortex

Individuals born without a cerebellar cortex but possessing a well-developed limbic system can certainly develop the neural network in the cerebellar nuclei into a CNN by integrating limbic signals, thereby enabling them to recognise patterns.

14.4 Third main theorem for biological neural networks

With the emergence of the hippocampus and the Papez loops, the transformation of cerebellar networks into feedback neural networks began. Through the introduction of a time-delayed feedback signal, the previous system state was fed into the cerebellar nuclei as an additional input value. The superimposition of past and present generated complex patterns that reflected the temporal structure of the signals and, for the first time, enabled predictive functions.