Andy Clark, Chris Thornton
The increasing level of attention being given to incremental learning, is, we argue, fully justified. Although some view the process as something akin to an 'efficiency hack' we argue that it is in fact a key cognitive process. The argument is based on a statistical observation. Learning, for most purposes, is all about acquiring the ability to generate appropriate outputs from given inputs, i.e., it is all about acquiring the ability to predict - or, in general, give a probability to - specific bindings of output variables. Such predictions can be justified in two quite different ways by available training data (i.e. input/output examples). They can be justified directly in virtue of being in 1-to-1 correspondence with probabilities observed in some re-coding of the data. Or they can be justified indirectly, in virtue of being in 1-to-1 correspondence with probabilities observed in some re-coding of the data. Thus while learning is driven by supplied training data, it must exploit some combination of these two types of justification.
Download compressed postscript file