The Associative Net
The simple neural network model of heterassociative memory known as the associative net provides the basis for investigating the effects of various biological constraints on associative memory performance.
The associative net (Willshaw et al, 1969) consists of a layer of input units connected to a layer of output units by feedforward connections. All unit activities are binary (0 or 1) and all connection weights are also binary. Pairs of patterns are stored in the net using a clipped Hebbian learning rule that changes a connection weight from 0 to 1 if both the input unit and output unit are active for the same pattern pair. The simplicity of this model allows analytical and numerical results to be obtained for finite sized nets using standard probability and information theory. Computer simulation of very large nets (in the order of thousands of input and output units) is also possible. We have used all these approaches in the following research.
Our work concerns the memory performance of this net when various biologically reasonable criteria are met. These criteria include:
Current work includes looking at memory performance when the
transmission of a signal from an input unit to an output unit is
probabilistic (Graham and Willshaw, 1997b,
corresponds to neurobiology - less than half of the action potentials
arriving at synapses in the mammalian hippocampus may elicit a
postsynaptic response. A stochastic net is formed by treating the
connection weights as probabilities of transmission. Instead of being
0 and 1, these weights may now be 0.2 and 0.8, for example. These are
the transmission probabilities at a synapse before and after
modification by Hebbian learning. Our results indicate that only
small differences between these probabilities are required to achieve
a functioning associative memory. Differences of the order of 0.4 or
less may be optimal if there is a cost associated with the magnitude
of the difference.
Graham, B. and Willshaw, D. (1997b) An associative memory model with probabilistic synaptic transmission. In Bower, J.M. (ed), Computational Neuroscience: Trends in Research, 1997, 315-319. Plenum Press. (manuscript).
Graham, B. and Willshaw, D. (1995b) Capacity and information efficiency of a brain-like associative net. In Tesauro, G., Touretzky, D., Leen, T. (eds), Neural Information Processing Systems 7, 513-520. MIT Press. (manuscript).
Buckingham, J. and Willshaw, D. (1993) On setting unit thresholds in an incompletely connected associative net. Network, 4, 441-459.
Buckingham, J. and Willshaw, D. (1992) Performance characteristics of the associative net. Network, 3, 407-414.
Willshaw, D., Buneman, O. and Longuet-Higgins, H. (1969) Non-holographic associative memory. Nature, 222, 960-962.