Entropy & the Second Law · Guide · glidepathscience.org
GuideEntropy & the Second Law
Contents
This Guide explains the ideas and equations behind every Walkthrough step, Concept Challenge question and Predict round. Part 1 prepares the Intro track and Part 2 the Advanced track; each takes about a quarter of an hour to read. Each section ends with a worked example that is not one of the sim’s own problems.
Part 1 · The essentials
Math you’ll need: algebra, powers of two and of ten, and the natural logarithm, ln; one optional box uses an integral.
Processes that run one way
Drop a little ink into a glass of water and it spreads until the water is evenly tinted; it never gathers itself back into a drop. A hot pan set on a cold counter cools while the counter warms; the reverse never happens. Film either one and run the film backward, and anyone can tell. Yet the laws that govern each molecule do not care which way time runs. When two molecules collide elastically, the same collision played backward is another perfectly good elastic collision, with the same speeds and the same total energy.
So the one-way character of everyday processes is not written into the laws of motion. It comes from counting. A gas that has spread through a box can be arranged in an enormous number of ways that all look “spread out”, and in a vanishingly small number of ways that look “gathered on one side”. Every collision shuffles the molecules from one arrangement to another, and the shuffling lands, overwhelmingly, in the kind of arrangement that is most common. This section and the next ones make that counting precise. The quantity that does the counting is entropy, and the statement that a system left alone does not go back is the second law of thermodynamics.
Microstates and macrostates
Coins make the counting easy to see. Flip four coins. A complete description, such as HTTH, says how each coin landed: it is a microstate. Usually we care about less: how many heads came up. That partial description is a macrostate. Each coin has two outcomes, so $N$ coins have $2^N$ microstates, all equally likely for fair coins:
$Ω = 2^N$ (all microstates of $N$ coins)
The macrostates are not equally likely, because they contain different numbers of microstates. The number of microstates in a macrostate is its multiplicity, written $Ω$ (omega). For $n$ heads out of $N$ coins it is the number of ways to choose which $n$ coins show heads:
$Ω(N, n) = N!/[n!\,(N − n)!]$
This count is often written $C(N, n)$ and read “$N$ choose $n$”. The chance of a macrostate is its multiplicity divided by the total, $Ω/2^N$. All heads has a multiplicity of 1; the macrostates near half heads have the most. A common slip is to think that a “random-looking” sequence such as HTHHTTHT is more likely than HHHHHHHH. Each exact sequence is one microstate with the same chance; what is more likely is the macrostate “half heads”, because so many sequences belong to it.
Molecules in a box behave like coins. If each molecule is equally likely to be in the left or the right half, then “how many are on the left” is a macrostate, and its multiplicity is the same $Ω(N, n)$. With twenty molecules the count most often sits near ten and wanders a few either side: ten exactly turns up only 17.6 % of the time, but eight to twelve about three quarters of the time.
Why a gas spreads and never unmixes
Start with every molecule in the left half, behind a partition, and lift the partition. The molecules wander into the right half, and the fraction on the left falls from 1 toward ½. It does not stop exactly at ½: it keeps fluctuating, by a few percent for forty molecules and by less for more. But it never climbs back to 1. Nothing forbids it; it is just outnumbered.
Each molecule, moving independently, is in the left half half of the time. The chance that all $N$ of them are there at the same moment is ½ multiplied by itself $N$ times:
chance all on one side $= (½)^N$
Put another way, all the molecules are on the left once in $2^N$ looks. Each extra molecule doubles the wait; doubling the number of molecules squares it. Twenty molecules gather on one side about once in a million looks; a hundred, once in $1.27 × 10^{30}$; a mole of gas, once in a number with $1.8 × 10^{23}$ digits. That is why the spreading is called irreversible, even though every collision is reversible.
Entropy counts arrangements
Multiplicities are inconveniently large, and when two systems are put together their multiplicities multiply: every microstate of one can go with every microstate of the other, so $Ω = Ω_1Ω_2$. Ludwig Boltzmann defined a quantity that grows with the multiplicity but adds instead, the entropy:
$S = k_B\ln Ω$
Here $\ln$ is the natural logarithm and $k_B$ = 1.380649 × 10⁻²³ J/K is Boltzmann’s constant, which gives entropy its units, joules per kelvin. Because $\ln(Ω_1Ω_2) = \ln Ω_1 + \ln Ω_2$, the entropy of two systems together is $S_1 + S_2$. Ten coins at five heads have $Ω$ = 252, $\ln Ω$ = 5.53 and $S$ = 7.63 × 10⁻²³ J/K: tiny, because ten coins are few. A mole has about 6 × 10²³ particles, and its entropy comes out in joules per kelvin.
A gas let into a larger volume shows the formula at work. If the volume grows by a factor $V_f/V_i$, each molecule has that many times as many places to be, so $Ω$ grows by $(V_f/V_i)^N$ and the entropy by $Nk_B\ln(V_f/V_i)$. Writing $N = nN_A$ and $R = N_Ak_B$ = 8.314 J/(mol·K):
$ΔS = nR\ln(V_f/V_i)$
When a gas expands into a vacuum (a free expansion), no heat flows and an ideal gas’s temperature does not change; the entropy rises all the same, because the arrangements multiplied. Entropy measures how many ways a macrostate can happen, not how much heat went in.
Two blocks share energy: the second law
Heat flowing from hot to cold is the same counting in disguise. A simple model of a solid treats each atom’s vibrations as oscillators that hold energy in equal steps, or quanta, each of size $ε$. A block of $N$ oscillators holding $q$ quanta can share them out in many ways, and the more quanta it has, the more ways. When two blocks touch, quanta hop at random between their oscillators. Every arrangement of all the quanta over both blocks is equally likely, so the energy ends up in the split with the most arrangements.
That split is not “equal energy”. It gives every oscillator, in either block, the same share of quanta on average, so a block with twice the oscillators ends up with twice the energy:
$⟨q_A⟩ = qN_A/(N_A + N_B)$
Two blocks that no longer exchange energy on average are at the same temperature: that is what temperature means. A block that starts with more quanta per oscillator than its partner is hotter, and the random hops carry energy from it to the colder one until the shares match.
This is the second law of thermodynamics: the total entropy of an isolated system does not decrease. It is a statement about chances. The total can dip by a little for a moment, as the stage shows with small blocks, but with the 10²³ particles of a real object, a fall of any size you could measure is so unlikely that it never happens. Notice the word total: one part can lose entropy while another gains more.
Heat, temperature and ΔS = Q/T
When the counting is done for a body much larger than whatever it exchanges energy with, a reservoir, its temperature hardly changes, and the result is simple. Heat $Q$ entering a reservoir at temperature $T$ (in kelvin) raises its entropy by
$ΔS = Q/T$ (a reservoir)
with $Q$ positive for heat in and negative for heat out. The same formula holds for any system that takes in heat slowly at a steady temperature, such as ice melting at 0 °C, which takes in $Q = mL$ with $L$ its latent heat of fusion:
$Q = mL$ (melting)
Dividing by $T$ explains why heat flows one way. When heat $Q$ leaves a hot reservoir at $T_h$ and enters a cold one at $T_c$, the hot one loses $Q/T_h$ and the cold one gains $Q/T_c$, which is larger. The total rises. Heat flowing from cold to hot would lower it, so it does not happen on its own; a refrigerator can make it happen only by turning work into extra heat for the warm side. The Heat Engines & Thermodynamic Cycles sim follows that idea to engines and Carnot’s limit.
Keep the system and the surroundings apart when you judge a sign. Melting ice gains entropy; coffee cooling in a room loses it, while the room gains more; a gas compressed slowly at a steady temperature gives out heat and loses entropy, while the reservoir gains exactly as much. Only the total of system and surroundings never falls.
The rubber band
A rubber band is a tangle of long, flexible molecular chains. Each chain can wriggle into an enormous number of crumpled shapes, but into very few nearly straight ones. Stretching the band straightens the chains and lowers their entropy; that is why a band pulls back. Its tension does not come mainly from stretched chemical bonds, as a steel spring’s does, but from the chains’ tendency to return to their many crumpled arrangements.
Two surprising results follow. First, stretch a band quickly, too fast for heat to leave, and touch it to your lip: it is warmer. The chains lost entropy, and with no heat leaving, the band’s total entropy stays the same, so the thermal jiggling of its atoms must gain what the chains lost: the band warms. Let it cool while stretched, then release it quickly, and it cools below room temperature. Second, hang a weight on a band and warm the band with a hair dryer: the weight rises. A chain’s pull at a given stretch is proportional to its temperature, so a warmer band pulls harder and shortens. A steel spring does neither.
Part 2 · Going further
Math you’ll need: logarithms and factorials, partial derivatives, and the exponential function; one section uses the inverse hyperbolic tangent.
Einstein solids and Stirling’s approximation
Part 1’s blocks are Einstein solids: $N$ oscillators of one frequency, each holding a whole number of quanta $ε = hf$. The quanta are indistinguishable, so a microstate is just how many each oscillator holds. Write one as $q$ dots and $N − 1$ bars, the bars separating the oscillators: “●● | ● | ●●●” is three oscillators holding 2, 1 and 3. A microstate is a choice of which of the $q + N − 1$ symbols are dots:
$Ω(N, q) = (q + N − 1)!/[q!\,(N − 1)!]$
Counting the quanta as labeled ($N^q$) or using $N$ bars instead of $N − 1$ are the usual slips. The numbers grow fast: 400 oscillators with 400 quanta have about 10²³⁹ microstates. Their logarithms are what matter, and for large $n$, Stirling’s approximation gives them:
$\ln N! ≈ N\ln N − N$
Adding $\tfrac12\ln 2πN$ makes it far better. With it, $\ln Ω$ for an Einstein solid with many oscillators and quanta becomes $(q + N)\ln(q + N) − q\ln q − N\ln N$.
The sharp peak
Two Einstein solids sharing $q$ quanta have $Ω_{total}(q_A) = Ω_A(q_A)\,Ω_B(q − q_A)$ microstates with $q_A$ in A. The product peaks where its logarithm does, which for large solids is where $q_A/N_A = q_B/N_B$: equal quanta per oscillator. Because the fundamental assumption gives every microstate the same chance, $Ω_{total}(q_A)$ divided by the total is the probability of each split.
The peak is narrow, and it gets narrower in relative terms as the solids grow. For two equal solids at the same quanta per oscillator, the spread of $q_A$ grows only as $\sqrt N$ while its mean grows as $N$, so the relative width scales as $1/\sqrt N$. A hundredfold increase in size makes the peak ten times sharper; for a mole of oscillators the relative width is about 10⁻¹¹. That is why thermodynamics can treat the most likely macrostate as the only one: the others are never measured.
Temperature from counting
At the peak, moving one quantum from A to B changes $S_{total}$ by nothing: the entropy A loses equals the entropy B gains. In the limit of many quanta the condition is $∂S_A/∂U_A = ∂S_B/∂U_B$, and the quantity that is equal for two bodies in thermal equilibrium is the temperature. So the statistical definition is
$1/T = ∂S/∂U$
A body whose entropy rises steeply with energy is cold: a little energy buys it many new microstates, so it takes energy from a body whose entropy rises slowly. For an Einstein solid, adding one quantum multiplies $Ω$ by $(q + N)/(q + 1)$, so $ΔS = k_B\ln[(q + N)/(q + 1)]$ for $ΔU = ε$, and
$k_BT/ε = 1/\ln[(q + N)/(q + 1)]$
Inverting this gives the energy as a function of temperature, and its derivative the heat capacity. At high temperature, $k_BT ≫ ε$, each oscillator holds $k_BT$ on average and the heat capacity tends to $Nk_B$; a solid has three oscillators per atom, so $C → 3R$ = 24.9 J/(mol·K), the law of Dulong and Petit. At low temperature the quanta “freeze out” and $C$ falls toward zero, which the classical picture could not explain.
The entropy of a gas, of mixing, and Gibbs’ paradox
For an ideal gas the counting gives the Sackur–Tetrode equation, $S = Nk_B\{\ln[(V/N)/λ^3] + \tfrac52\}$, where $λ = h/\sqrt{2πmk_BT}$ is the thermal wavelength. At fixed energy only the $\ln V$ term changes, so a free expansion gives $ΔS = nR\ln(V_f/V_i)$, as in Part 1. Two different gases at the same temperature and pressure, mixed by lifting a partition, each expand into the whole volume; for amounts $n_i$ with mole fractions $x_i = n_i/n$,
$ΔS_{mix} = −R\sum n_i\ln x_i$
which is largest for an even mix, $nR\ln 2$. If the two sides hold the same gas, nothing measurable happens when the partition lifts, and putting it back restores the start, so $ΔS$ must be zero. Treating the molecules as distinguishable gives $nR\ln 2$ instead: Gibbs’ paradox. The cure is to count identical molecules as identical, dividing $Ω$ by $N!$, which is the $N$ inside the logarithm of Sackur–Tetrode and what makes entropy extensive.
The Boltzmann factor
Single out one small system, such as a single oscillator, in contact with a large reservoir at temperature $T$. When the small system holds energy $E$, the reservoir has $U − E$, and since the reservoir’s entropy falls by $E/T$, its multiplicity falls by $e^{−E/k_BT}$. Every microstate of the whole is equally likely, so the chance of the small system being in a particular state of energy $E$ is proportional to the reservoir’s multiplicity:
$P(E) ∝ e^{−E/k_BT}$
This is the Boltzmann factor, the starting point of statistical mechanics. Each additional quantum $ε$ costs the same factor $e^{−ε/k_BT}$, so an oscillator’s chances fall geometrically with its number of quanta. Low-energy states are not favored by any force: a reservoir simply has more microstates when it keeps the energy.
The entropic force
The simplest model of a rubber molecule is a one-dimensional freely jointed chain: $N$ links of length $b$, each pointing forward or backward. With $n_+$ forward and $n_−$ backward, the length is $L = (n_+ − n_−)b$ and the multiplicity is $C(N, n_+)$. With Stirling, the entropy per link at stretch $x = L/Nb$ is $s(x) = \ln 2 − \tfrac12[(1 + x)\ln(1 + x) + (1 − x)\ln(1 − x)]$ in units of $k_B$, falling from $\ln 2$ when slack to 0 when straight. Held at fixed temperature, the chain pulls with $F = −T\,∂S/∂L$:
$F = (k_BT/b)\,\mathrm{artanh}(L/Nb)$
For small stretches this is a spring, $F ≈ k_BTL/(Nb^2)$, with a stiffness proportional to $T$; near full length it grows without limit. Stretched fast, the chain and the band’s lattice (heat capacity $C = c\,Nk_B$) keep their total entropy, $Nk_Bs(x) + C\ln T$, so
$\ln(T_2/T_1) = [s(x_1) − s(x_2)]/c$
An ideal chain stores no energy of its own, so all the work done stretching it becomes heat. The sim’s lattice value, $c$ = 20, is illustrative: it makes a fast stretch warm the band by a few kelvin, as real rubber does.
The arrow of time
Josef Loschmidt pointed out that if the laws of motion are reversible, reversing every velocity of a gas that has spread must send it back to where it started, lowering its entropy. He was right: reversed exactly, the gas retraces its path. The second law is not a law of motion but a statement about which microstates are common. The reversed state is one of the extraordinarily rare microstates that lead somewhere special, and in practice no reversal is exact: in a gas, collisions amplify the tiniest error exponentially, so a reversal made late returns only part of the way. Henri Poincaré showed that a closed system will eventually return close to its start, but for a hundred molecules looked at ten times a second the wait for all of them to be on one side is $4 × 10^{21}$ years, against the universe’s $1.4 × 10^{10}$.
Why, then, does entropy rise toward the future and not the past? Because the universe started in a state of very low entropy, and everything since has been running downhill from it. The arrow of time is a fact about where things started, not about the laws.
Reference
Common mistakes
Part 1
- Calling one sequence of flips “more random” and so more likely: every microstate is equally likely; macrostates differ.
- Thinking unmixing is forbidden: it is allowed, and outnumbered by $2^N$ to 1.
- Adding multiplicities instead of multiplying them; entropies add.
- Using log base 10 in $S = k_B\ln Ω$.
- Thinking heat flows from more energy to less: it flows from higher temperature to lower.
- Reading the second law as “every object’s entropy rises”: only the total of system and surroundings never falls.
- Using °C in $ΔS = Q/T$; the temperature is in kelvin.
- Treating rubber as a spring: its pull is entropic and grows with temperature.
Part 2
- Counting quanta as labeled ($N^q$) in an Einstein solid.
- Expecting the relative width to fall as $1/N$; it falls as $1/\sqrt N$.
- Taking $k_BT = U/N$ for an Einstein solid at low temperature.
- Using $nR\ln 2$ for an uneven mix, or for a gas mixed with itself.
- Using the linear spring for a chain near full stretch.
The equation sheet
| Part 1 | |
|---|---|
| microstates of coins | $Ω = 2^N$ (all microstates of $N$ coins) |
| multiplicity | $Ω(N, n) = N!/[n!\,(N − n)!]$ |
| all on one side | chance all on one side $= (½)^N$ |
| entropy | $S = k_B\ln Ω$ |
| free expansion | $ΔS = nR\ln(V_f/V_i)$ |
| a reservoir | $ΔS = Q/T$ (a reservoir) |
| melting | $Q = mL$ (melting) |
| sharing energy | $⟨q_A⟩ = qN_A/(N_A + N_B)$ |
| Part 2 | |
|---|---|
| Einstein solid | $Ω(N, q) = (q + N − 1)!/[q!\,(N − 1)!]$ |
| Stirling | $\ln N! ≈ N\ln N − N$ |
| temperature | $1/T = ∂S/∂U$ |
| Einstein solid’s $T$ | $k_BT/ε = 1/\ln[(q + N)/(q + 1)]$ |
| mixing | $ΔS_{mix} = −R\sum n_i\ln x_i$ |
| Boltzmann factor | $P(E) ∝ e^{−E/k_BT}$ |
| entropic force | $F = (k_BT/b)\,\mathrm{artanh}(L/Nb)$ |
| a fast stretch | $\ln(T_2/T_1) = [s(x_1) − s(x_2)]/c$ |
Symbols: $N$ the number of coins, molecules, oscillators or links; $n$ heads (or moles); $Ω$ the multiplicity; $S$ the entropy; $k_B$ = 1.380649 × 10⁻²³ J/K; $R$ = 8.314 J/(mol·K); $V$ a volume; $Q$ heat in; $T$ the temperature in kelvin; $m$ a mass and $L$ a latent heat; $q$ quanta of energy $ε$ each; $x_i$ mole fractions; $E$ an energy; $b$ a link’s length, $L$ the chain’s length and $x = L/Nb$; $s(x)$ the entropy per link in units of $k_B$; $c$ the lattice’s heat capacity per link in units of $k_B$.
Sources and further reading
- OpenStax, University Physics Volume 2, chapter 4, “The Second Law of Thermodynamics,” its sections on entropy and on entropy on a microscopic scale, CC BY-NC-SA 4.0 (openstax.org).
- T. A. Moore and D. V. Schroeder, “A different approach to introducing statistical mechanics,” American Journal of Physics 65, 26 (1997), and D. V. Schroeder, An Introduction to Thermal Physics, for Einstein solids sharing energy.
- C. Kittel and H. Kroemer, Thermal Physics, 2nd ed. (Freeman, 1980), for the one-dimensional chain.
- This sim’s Field Manual (H) for the model and its limits; Heat Engines & Thermodynamic Cycles for engines, refrigerators and Carnot’s limit; Kinetic Theory & Gas Laws for the gas laws and molecular speeds.
Entropy & the Second Law v1.1 · © 2026 Dr. David Meier · CC BY-NC-SA 4.0 · glidepathscience.org