Notes on Entropy
I found this really interesting passage in an 1870 paper by Boltzmann, lectures on gas theory:
We make the following remarks:
It is by no means the sign of the time which constitutes the characteristic difference between an ordered and a disordered state. If, in the “initial states” of the mechanical picture of the world, one reverses the directions of all velocities, without changing their magnitudes or the positions of the parts of the system; if, as it were, one follows the states of the system backwards in time, then he would likewise first have an improbable state, and then reach ever more probable states. Only in those periods of time during which the system passes from a very improbable initial state to a more probable later state do the states change in the positive time direction differently than in the negative.
The transition from an ordered to a disordered state is only extremely improbable. Also, the reverse transition has a definite calculable (though inconceivably small) probability, which approaches zero only in the limiting case when the number of molecules is infinite. The fact that a closed system of a finite number of molecules, when it is initially in an ordered state and then goes over to a disordered state, finally after an inconceivably long time must again return to the ordered state,* is therefore not a refutation but rather indeed a confirmation of our theory. One should not however imagine that two gases in a liter container, initially unmixed, will mix, then again after a few days separate, then mix again, and so forth. On the contrary, one finds by the same principles which I used5 for a similar calculation that not until after a time enormously long compared to 101010 years will there be any noticeable unmixing of the gases. One may recognize that this is practically equivalent to never, if one recalls that in this length of time, according to the laws of probability, there will have been many years in which every inhabitant of a large country committed suicide, purely by accident, on the same day, or every building burned down at the same time—yet the insurance companies get along quite well by ignoring the possibility of such events. If a much smaller probability than this is not practically equivalent to impossibility, then no one can be sure that today will be followed by a night and then a day.
We have looked mainly at processes in gases and have calculated the function H for this case. Yet the laws of probability that govern atomic motion in the solid and liquid states are clearly not qualitatively different in this respect from those for gases, so that the calculation of the function H corresponding to the entropy would not be more difficult in principle, although to be sure it would involve greater mathematical difficulties.
Relation to the second law of thermodynamics.
If therefore we conceive of the world as an enormously large mechanical system composed of an enormously large number of atoms, which starts from a completely ordered initial state, and even at present is still in a substantially ordered state, then we obtain consequences which actually agree with the observed facts; although this conception involves, from a purely theoretical—I might say philosophical—standpoint, certain new aspects which contradict general themodynamics based on a purely phe-nomenological viewpoint. General thermodynamics proceeds from the fact that, as far as we can tell from our experience up to now, all natural processes are irreversible. Hence according to the principles of phenomenology, the general thermodynamics of the second law is formulated in such a way that the unconditional irreversibility of all natural processes is asserted as a so-called axiom, just as general physics based on a purely phenomenological standpoint asserts the unconditional divisibility of matter without limit as an axiom.
Just as the differential equations of elasticity theory and hydrodynamics based on this latter axiom will always remain the basis of the phenomenological description of a large group of natural phenomena, since they provide the simplest approximate expression of the facts, so likewise will the formulas of general thermodynamics. No one who has fallen in love with the molecular theory will approve of its being given up completely. But the opposite extreme, the dogma of a self-sufficient phenomenology, is also to be avoided. Just as the differential equations represent simply a mathematical method for calculation, whose clear meaning can only be understood by the use of models which employ a large finite number of elements, so likewise general thermodynamics (without prejudice to its unshakable importance) also requires the cultivation of mechanical models representing it, in order to deepen our knowledge of nature–not in spite of, but rather precisely because these models do not always cover the same ground as general thermodynamics, but instead offer a glimpse of a new viewpoint. Thus general thermodynamics holds fast to the invariable irreversibility of all natural processes. It assumes a function (the entropy) whose value can only change in one direction—for example, can only increase—through any occurrence in nature. Thus it distinguishes any later state of the world from any earlier state by its larger value of the entropy. The difference of the entropy from its maximum value— which is the goal of all natural processes—will always decrease. In spite of the invariance of the total energy, its transformability will therefore become ever smaller, natural events will become ever more dull and uninteresting, and any return to a previous value of the entropy is excluded.
One cannot assert that this consequence contradicts our experience, for indeed it seems to be a plausible extrapolation of our present knowledge of the world. Yet, with all due recognition to the caution which must be observed in going beyond the direct consequences of experience, it must be granted that these consequences are hardly satisfactory, and the discovery of a satisfactory way of avoiding them would be very desirable, whether one may imagine time as infinite or as a closed cycle. In any case, we would rather consider the unique directionality of time given to us by experience as a mere illusion arising from our specially restricted viewpoint.
Application to the universe.
Is the apparent irreversibility of all known natural processes consistent with the idea that all natural events are possible without restriction? Is the apparent unidirectionality of time consistent with the infinite extent or cyclic nature of time? He who tries to answer these questions in the affirmative sense must use as a model of the world a system whose temporal variation is determined by equations in which the positive and negative directions of time are equivalent, and by means of which the appearance of irreversibility over long periods of time is explicable by some special assumption. But this is precisely what happens in the atomic view of the world.
One can think of the world as a mechanical system of an enormously large number of constituents, and of an immensely long period of time, so that the dimensions of that part containing our own “fixed stars” are minute compared to the extension of the universe; and times that we call eons are likewise minute compared to such a period. Then in the universe, which is in thermal equilibrium throughout and therefore dead, there will occur here and there relatively small regions of the same size as our galaxy (we call them single worlds) which, during the relative short time of eons, fluctuate noticeably from thermal equilibrium, and indeed the state probability in such cases will be equally likely to increase or decrease. For the universe, the two directions of time are indistinguishable, just as in space there is no up or down. However, just as at a particular place on the earth’s surface we call “down” the direction toward the center of the earth, so will a living being in a particular time interval of such a single world distinguish the direction of time toward the less probable state from the opposite direction (the former toward the past, the latter toward the future). By virtue of this terminology, such small isolated regions of the universe will always find themselves “initially” in an improbable state. This method seems to me to be the only way in which one can understand the second law—the heat death of each single world—without a unidirectional change of the entire universe from a definite initial state to a final state.
Obviously no one would consider such speculations as important discoveries or even— as did the ancient philosophers—as the highest purpose of science. However it is doubtful that one should despise them as completely idle. Who knows whether they may not broaden the horizon of our circle of ideas, and by stimulating thought, advance the understanding of the facts of experience?
That in nature the transition from a probable to an improbable state does not take place as often as the converse, can be explained by assuming a very improbable initial state of the entire universe surrounding us, in consequence of which an arbitrary system of interacting bodies will in general find itself initially in an improbable state. However, one may object that here and there a transition from a probable to an improbable state must occur and occasionally be observed. To this the cosmological considerations just presented give an answer. From the numerical data on the inconceivably great rareness of transition from a probable to a less probable state in observable dimensions during an observable time, we see that such a process within what we have called an individual world—in particular, our individual world—is so unlikely that its observability is excluded. In the entire universe, the aggregate of all individual worlds, there will however in fact occur processes going in the opposite direction. But the beings who observe such processes will simply reckon time as proceeding from the less probable to the more probable states, and it will never be discovered whether they reckon time differently from us, since they are separated from us by eons of time and spatial distances $10^{10^{10}}$ times the distance of Sirius– and moreover their language has no relation to ours.*
Very well, you may smile at this; but you must admit that the model of the world developed here is at least a possible one, free of inner contradiction, and also a useful one, since it provides us with many new viewpoints. It also gives an incentive, not only to speculation, but also to experiments (for example on the limit of divisibility, the size of the sphere of action, and the resulting deviations from the equations of hydrodynamics, diffusion, and heat conduction) which are not stimulated by any other theory.