This is one of those kid’s questions, where most adults will say something like “stupid boy, Pike!” and move to something else. After all, it’s obvious, isn’t it? Hot things cool down, cold things warm up, and we know the laws of thermodynamics that describe what happens. Heat always moves from hotter to colder, and always has done (even in refrigerators where we manipulate the pressures and thus temperatures, the heat flow is only from something hotter at that time to something cooler). Nothing to see here, move on….
It’s not however such a silly question, and the answer is more complex than it appears. In fact, heat does not move from hotter to colder when we look at what’s actually happening at the microscopic level, and it only appears to have done so when we look at the temperature distribution after some time has passed. Heat is kinetic energy, and we can either look at the movements of the particles that are moving or we can follow what happens to the energy as it moves. It is however random in its direction, and mostly will follow a standard distribution in its range of energy-levels or velocities except in special cases. I’ve put a somewhat-explanatory picture in, showing the paths of selected molecules in a gas. Each collision will give the gas-molecule a new velocity, and both the energy-level (speed) and the direction will be random after each collision. For some nice explanations and pictures of random walks, see the wiki.
The thing to note here is that even though there may be hotter and colder places in this gas (this is not a drawing of an equilibrium situation), the direction those molecules go doesn’t depend on the temperature. A molecule may go towards a hot-spot or away with the same probability – the temperature in that location has absolutely no effect on it. The molecule has no way of predicting what the temperature will be along the new path it takes (or indeed when it will collide and with what – it’s random), and also doesn’t know what the temperature was where it came from. All it has is its own kinetic energy, and the size of that energy and its direction are random. In a gas, at least, heat (as in the kinetic energy a molecule has) will thus travel from colder to hotter nearly as often as it goes the other way. The actual temperature at any point has no effect on this. To work out the probabilities of heading towards a hotter spot or a colder one, all we need to know for a particular collision is what solid angle points towards hotter and what solid angle points towards colder. There is no biasing of the probability by the temperature in any direction, and we’re just looking at a random process. If we try to say that it is otherwise, then we also need to posit some way that a molecule can predict the future and has memory of its past, and will make a decision accordingly. That seems logically inadmissible (and would make Pool or Snooker an impossible game, too).
The molecule will follow a random walk, and over a sufficient period it will have equal probability of being anywhere in the box that contains the gas. Much the same happens to the kinetic energy that is shared amongst the molecules, though since at each collision the relative quantities of energy each molecule comes in with and goes out with will get redistributed, the random walk is a little more complex to follow. The random nature, however, means that any bit of energy you put into that box will spread out from that location until it has an equal probability of being in every location it can reach. That energy can and will at times move towards an area of higher concentration (a hotter place) and at other times towards a colder place. It’s a random walk, which over time covers a larger and larger area.
Gases and gas molecules are fairly easy to understand, but in a solid we have much the same thing happening. Here instead we have wave-motions of atoms vibrating around their rest lattice positions, and these waves can be treated as energy-packets (phonons) that have some properties of particles. In the same way, though, those phonons carry no information about what temperature they’ve come from or are heading towards – it’s just a moving packet of energy that suffers random collisions and changes of direction. They will follow the same random walk and for each energy-packet we’ll see the probability gradually equalise for it being in any location it can reach.
If we’ve left a long-enough time from when we put the extra energy in at one point in the object, then that energy will have essentially equal probabilities of being anywhere. Since all the other energy-packets have the same equal probabilities of being everywhere, then everything will be at the same temperature since it has the same average energy-level.
When we look at things on a large scale, therefore, heat will move from hotter to colder because that random walk leads to an equal probability of being everywhere. At the atomic scale, however, the temperature is actually irrelevant in defining which direction the kinetic energy will move. The movements we are talking about also will not suddenly stop when thermal equilibrium is achieved. They continue all the time, and you can verify this with a microscope and some pollen as Brown did in 1827 (Brownian motion).
What we call thermal equilibrium is thus somewhat of a fiction, since it remains full of dynamic movement whether of molecules, phonons or other kinetic energy. Kinetic energy, after all, has to be moving in order to measure it. It’s also pretty obvious that heat will move from colder to hotter exactly as often when we are in thermal equilibrium, since we’re dealing with random systems and there will be very local fluctuations around that average energy all the time. Temperature cannot be an instantaneous measurement, though is is often represented as such, but instead is an average energy measurement over a period that cannot be zero.
That we see that “heat always goes from hotter to colder” is an emergent property of a very large number of random walks at a rate that our normal thermometers cannot distinguish. If we have equipment that can resolve the individual movements, though, we get a totally different picture where the temperature is an irrelevant measurement and instead we need to look at the energy-vectors and momentum transfers. For single energy-transactions (collisions of individual molecules or phonons) the 2LoT is actually irrelevant, and only comes into play when we are looking at large numbers of such random transactions.