Algorithmic information theory (AIT) provides a neat way of approaching Occam's Razor. If we have some result or set of results to be explained we can identify theories to explain them with programmes (or algorithms) which output that result (in some encoded form). Now Occam's Razor can be interpreted as stating that the "best" theory is the one with the shortest programme. In other words, this is a formulation of theoretical science as the compression of the information nature provides.

It is a remarkable fact that much of nature is highly compressible, or at least that approximate versions of nature are - for example, the entire of rigid body mechanics at a human scale can be expressed in Newton's three short laws, and as I understand it basically the entire of quantum mechanics is contained in one little partial differential equation - Schrodinger's equation. Of course, we can't actually know whether any theory we have actually is a compression of nature as it really is, or is just a compression of the approximation of nature we can measure. So it is quite conceivable that nature is in fact completely incompressible - random, in the AIT sense - in which case Occam's Razor could not be applied without shaving off some explanatory power, and the best theory would simply be "what is is what is".