Slimy, Hilly, and Pixy

In cognitive (neuro)science, parallel distributed processing (PDP) models are often called neural network models, and the network process is often verbalized in neural terms such as activation spreading, excitation, and inhibition. This neural metaphor is understandable but also somewhat misleading: PDP is a strong modeling tool, PDP models of cognition are not necessarily closer to the real thing than other types of models are. The metaphor also obscures the fact that PDP does not require neurons but is based on simple natural principles which, below, are illustrated in animate, inanimate, and artificial contexts.


Animate PDP

Nakagaki, Yamada, and Tóth (2000) won the 2008 Ig Nobel Prize for Cognitive Science with a fine example of simple natural PDP principles. They investigated "problem solving" by the slime mould Physarum polycephalum, as follows:

Physarum pieces in 4x4 cm maze with agar-filled paths	Once released, Physarum spreads and fills the maze
Then, two agar blocks are placed, with four connecting routes	After eight hours, Physarum has selected a shortest route

Hence, the slime mould can be said to perform parallel distributed processing to find the shortest path between the two food sources. By filling the maze, it gets a distributed representation of the maze paths and, after simultaneous processing at all places in the maze, it comes up with a solution.

On the one hand, Nakagaki et al. suggested that this maze-solving ability might be a form of primitive intelligence, but this suggestion can hardly be taken seriously. The slime mould does not have a brain or nervous system and, if intelligent behavior means anything, it implies that the maze problem becomes more difficult as a function of the number of paths through the maze. To the slime mould, however, the time to solve the maze most likely increases as a function of the size of the maze. This implies that the slime mould acts either super-intelligently or, more likely, mindlessly.

On the other hand, Nakagaki et al. also gave a more serious physical explanation which relies on proteins and nutrients that travel back and forth through the organism ("activation spreading") and that allow the organism to change shape by shrinkage ("inhibition") of poorly fed dead-end parts and by expansion ("excitation") of richly fed through parts.


Inanimate PDP

By simply following the laws of physics, even inanimate material can show "intelligent" behavior. Here is the example used in Proceedings of the National Academy of Sciences USA 2004 (see also Cognitive Processing 2012) to illustrate that serial distributed processing on a computer can be translated to parallel distributed processing on other hardware. The fluid in the hilly tube system below finds a time-shortest path by way of parallel distributed processing:

At time T=0, a fluid is poured into node 0. The fluid is such that it hardens within one time unit after it stops flowing. Every link between two nodes is a soft tube that expands as the fluid enters and that consists of straight segments having slopes such that the fluid takes one time unit to cross one segment. Every node has a separate outlet for each outgoing tube, but only one inlet for all incoming tubes. An inlet has about the same cross section as one fluid-filled tube. Hence, when the fluid reaches the inlet through one or more tubes, the remaining tubes are automatically sealed off. Thus, at time T=1, the fluid reaches node 2, sealing off the tube between nodes 1 and 2.
At time T=2, the fluid has filled the dead-end tube between nodes 1 and 2, and the then nonflowing fluid therein has hardened at time T=3. By then, the fluid has also already reached node 5.
Around time T=4, there is still some filling of dead-end tubes and hardening of the fluid therein but, as of time T=5, the only remaining flow is through a time-shortest path, i.e., a path consisting of a minimal number of segments.

Hence, the number of possible paths in such a hilly tube system on N nodes may be in the order of 2^N but, by way of parallel distributed processing, the fluid takes only N time steps to single out the shortest path. This hilly tube system is in fact a PDP implementation of Dijkstra's (1959) shortest path method (SPM). The SPM is a so-called smart computer algorithm that, by way of serial distributed processing, takes in the order of N² time steps to single out a shortest path.

The SPM performs operations that are quite similar to what the hilly tube system does. It needs more time steps, but this is just because it does serially what the hilly tube system does in parallel during one time step. The same holds for computer simulations of PDP models. This serial-instead-of-parallel aspect, however, does not alter the underlying principle of distributed processing (see also Smart processing).


Artificial PDP

Jeff Jones (Department of Computer Science, University of Chester, UK) kindly provided the next two videos that explore shortest path problem solving using physically inspired systems (here, diffusion and chemotaxis) to research image processing, perception and other related spatial problems.

Video 1 shows the spreading, from exit to entrance, of the diffusing wavefront of a chemical through a maze. Video 2 shows a population of simple agents ("pixies") that, starting at the entrance, move through the maze, "sensing" and moving towards the highest concentration gradient of the (now unseen) diffused chemical.

Start Video 2 about ten seconds after Video 1 to see that the pixies find an initial path and that the diffusion eventually leads to a change in the wavefront (at the center of the maze) after which the pixies switch to the shortest path.

Video 1 Chemical spreading through the maze		Video 2 Shortest path finding pixies