Occam versus von Helmholtz
or
the Bayesian duality of simplicity and likelihood

The simplicity-likelihood debate in vision is like the nature-nurture debate in biology. The Helmholtzian likelihood principle reflects the assumption that vision is driven by veridicality in the external world; this would imply, by definition, that vision is highly veridical in this world. In contrast, the Occamian simplicity principle reflects the assumption that vision is driven by efficient internal mechanisms; as indicated below, there is mathematical evidence suggesting that this implies that vision is fairly veridical in many different worlds.

The likelihood principle suggests that vision is a special-purpose system in that it is highly adapted to one specific world, and that proximal stimuli are interpreted on the basis of knowledge about probabilities in this world. Conversely, the simplicity principle suggests that vision is a general-purpose system in that it is fairly adaptive to many different worlds, and that the perceived structures in the world are interpretations on the basis of knowledge-free simplest descriptions of scenes. Hence, whereas the likelihood principle specifies knowledge as a resource of vision, the simplicity principle rather specifies vision as a source of knowledge.

These diametrically-opposed starting points can be compared in more detail by making a Bayesian distinction between viewpoint-independencies and viewpoint-dependencies in stimulus interpretations (see also Object versus viewer). The viewpoint-independency (or prior) indicates the goodness of the distal stimulus H as hypothesized in the interpretation, independently of the proximal stimulus D. The viewpoint-dependency (or conditional) indicates how well the proximal stimulus D fits in with the hypothesized distal stimulus H (this consistency relation between D and H is denoted by D|H).

The combination of priors and conditionals then indicates how well the hypothesized distal stimulus H fits in with the proximal stimulus D (this so-called posterior is denoted by H|D). Thus, as the next figure illustrates, the Helmholtzian likelihood principle can be said to lead to interpretations that maximize certainty by way of a (Bayesian) multiplication of prior and conditional probabilities in the world, whereas the Occamian simplicity principle can be said to lead to interpretations that minimize information by way of a summation of prior and conditional complexities as assessed by internal mechanisms.




The foregoing illustrates a formal duality which enables a more detailed comparison of the two principles. That is, as illustrated in the next figure, both principles can be formulated in Bayesian terms as well as in information-theoretic terms. The Helmholtzian real-world probabilities, on the one hand, may be converted into so-called surprisals (probabilistic quantifications of amounts of information) and then information may be minimized; this conversion from probabilities to amounts of information is characteristic of Shannon's (1948) classical information-theoretic approach. The Ocammian descriptive complexities, on the other hand, may be converted into so-called precisals (artificial probabilities) and then certainty may be maximized; this conversion from amounts of information to probabilities is characteristic of modern information-theoretic approaches.

	Bayes
von Helmholtz		Occam

These conversions imply that the two principles can be formulated by way of the same mathematical formulas. This, however, does not at all imply that the two principles are equivalent (as has been claimed in the literature). What matters to the equivalence question is which amounts of information or which probabilities are put in the formulas, and in this respect, the two principles differ fundamentally. In other words, the two optimization formulas may be equivalent mathematically, but this does not imply that the two principles are equivalent, simply because they rely on fundamentally different quantifications of amounts of information and probabilities.

Yet, a comparison of the two principles in Bayesian terms led to the finding that the two principles may be far apart regarding the viewpoint-independent priors but seem close regarding the viewpoint-dependent conditionals. This implies that, whereas the likelihood principle is by definition highly veridical in one world, the simplicity principle promises to be fairly veridical in many different worlds (see also Everyday perception). There is no proof that our specific world is among these many different worlds but this finding implies that, evolutionary, the simplicity principle is a serious contender -- after all, the evolution may have favoured its adaptivity to changing environments.


With respect to research practice in vision, the mathematical equivalence of the formulas implies that modellers are free to choose either amounts of information or probabilities to model visual phenomena. Then, however, they also have to be aware that this choice presupposes a choice between the likelihood and simplicity principles which, after all, determines which amounts of information or which probabilities are going to be used. Notice that the latter choice goes deeper than the pragmatic question of how well models fit data. A probabilistic model may fit human data, and the employed probabilities can therefore be said to comply with perceptual probabilities, but whether these probabilities comply with Helmholtzian probabilities or with Occamian precisals is a totally different question.

For instance, it is perfectly legitimate to model the result of the interpretation process in terms of probabilities. The book edited by Knill and Richards (1996) gives fine examples of such models. Despite appearances, however, it is questionable whether these models fit in with the likelihood paradigm. Fundamental problems with quantifying real-world probabilities are side-stepped (see Bertrand's paradox); the distinction between viewpoint-independencies and viewpoint-dependencies is fuzzy; and viewpoint-independent prior probabilities, especially, seem to be chosen on the basis of intuition or simplicity rather than on the basis of real frequencies of occurrence in the world.

In fact, from the beginning, the whole purpose of the simplicity paradigm has been to circumvent such problems. For instance, van Lier et al. (1994) launched an empirically successful model of the integration of viewpoint-independencies and viewpoint-dependencies in amodal completion (see also T-junctions). This model uses the coding model of SIT to quantify prior complexities of object shapes and conditional complexities of relative object positions. The conditional complexities agree well with intuitively assessed probabilities: as a rule, a relative position that is more complex descriptively is also more accidental intuitively. Such an intuitive assessment seems veridical in case of relative object positions, but there is no indication that it is also veridical in case of object shapes. In fact, if the visual system indeed is guided by simplicity, then this might explain that, intuitively, simple shapes seem to occur more frequently in the world, even if they do not.

Hence, if one wants to model human visual phenomena in terms of Helmholtzian probabilities, one faces the problem that these probabilities are hardly quantifiable, if at all. Then, employing Occamian precisals seems like a good alternative -- in which case one might model the phenomena just as well directly in terms of the underlying descriptive complexities, by the way. Then, however, one also should acknowledge that the explanatory principle is internal efficiency, not external veridicality.


For an extensive discussion on these issues, see Psychological Bulletin 2000
For a brief discussion on these issues, see In the Mind's Eye 2007
For an updated brief discussion on these issues, see Acta Psychologica 2011