Compression Contest and Marctar's Axe
Marcus Hutter has posted a 50,000 Euro prize contest for compressing (a subset of) Wikipedia. While I applaud Dr. Hutter for his generous contribution to stimulating AI research, and I do believe that cognition often can be viewed as compression (as championed by Gerard Wolff), I do have one reservation with this idea:
The idea that a smaller theory is a better theory can be traced at least to Ockham's Razor. When I applied The Cruncher to the frames of a (very low resolution, but continuitous) movie, I found that the best compression was when the 1st frame was described, then every frame was described in terms of the previous frame (much the way that mpegs work). However, I don't think this is how people do it. I don't record my day by remembering when I woke up then every instant in terms of the previous instant. If I did, I'd have to spend a good deal of time "unpacking" my day to tell you what I had for supper (and presumably less time to tell you what I had for breakfast).
Perhaps a more extreme example would be Euclidean Geometry. This can be compressed down to the 5 postulates (plus some inference rules), but I don't think anyone rederives a commonly used lemma every time they use it.
Therefore, I propose an alternate to Ockham's Razor, which I'll call Marctar's Axe: The best model is that which has the smallest average query time (in terms of steps of computation to answer it). By "query" I mean things like "How many times have you seen this pattern?" or "What is the usual outcome of this pattern?".
My suspicion is that compression will usually fall out of Marctar's Axe, but there will be some cases where Marctar's Axe will tell you to "cache" results and thereby trade in a lot of computation time for a little bit of memory.
