The Bactra Review An Introduction
to Natural Computation
It is not clear that Rissanen's description-length is the appropriate measure
of informational cost in biological computers. The description-length of a
piece of data x, with respect to a model M, in a class of
models M, is -log P(x|M) + D(M, M), where
P(x|M) is the probability of x given M, and
D(M, M) is the number of bits needed to specify M given a
coding-scheme for all the models in M. To use an example of Ballard's, our
models are neural networks, the description-length favors networks which make
the current data x highly probable, and it favors networks which have
short descriptions, given our coding-scheme. I do not, however, see that the
second, model-code, term has any biological relevance, even of the
tenuous sort belonging to the first, data-model term. We need to
represent, e.g., the synaptic weights in the networks somehow, but the brain
doesn't --- it just has to have the synapses! By analogy with William James's
``psychologist's
fallacy,'' we may call this mistake the neurobiologist's fallacy.