I chose the name The Chaostician for this blog. To live up to this name, this blog should be about chaos, in particular, what a physicist means when referring to ‘chaos theory’.
This seven eight part series is my explanation of chaos theory to a popular audience. Chaos is a mechanism that allows deterministic objects to behave unpredictably. I will explain why this happens and what kinds of predictions we can make when something is chaotic.
Those of you who are already experts in nonlinear dynamics know that there are many ways to approach chaos. In this series, I will focus on behavior on strange attractors in dissipative chaos. In particular, the goal is to explain the ideas underlying periodic orbit theory and trace formulas you can use to calculate expectation values. Other series will address other approaches to chaos.
Part III will be about Lyapunov exponents. If you search for a definition of what chaos theory is, the most common result is that chaos occurs whenever there is a positive Lyapunov exponent. While I would prefer defining chaos to be anything with positive topological entropy,[1]I will describe topological entropy in Part VII. Lyapunov exponents are extremely important. They explain what we mean when we say that something ‘behaves unpredictably’.
Part I: Introduction.
Part II: The Simplest Chaotic System.
Part III: Lyapunov Exponents.
Part IV: Strange Attractors.
Part V: Continuous Time.
Part VI: Stretch and Fold.
Part VII: Partitions and Symbols.
Part VIII: Periodic Orbit Theory.
Prerequisites: High school level math, in particular, exponential growth and logarithms. Also a limit. This is the post of the series that involve the most calculations. I have pushed some of the calculations to an appendix that uses geometry and trigonometry, which you can skip if you are willing to trust that I got the result correctly. If you don’t understand anything, please ask – in the comments or by emailing thechaostician@gmail.com.
Originally Written: January 2017.
Confidence Level: Established science since the mid twentieth century. Occasional philosophy mixed in.
Sensitive Dependence on Initial Conditions
Think back to the videos in the last post for the motion of the logistic map when $r = 3.65$ and $r = 4$. The two initial conditions start out at 0.1 and 0.101 – very close together. They do not stay close together. The distance between them grows until their motion appears to be completely independent of each other.
This is a typical phenomenon in chaotic systems. If you start at some initial location, you can figure out what the motion will be. Or you could also start at a different initial location extremely close to the first one. If you start close enough, then you might expect that the resulting motion will be similar. This is only true for a short amount of time. As time goes on, the distance between them grows exponentially. Eventually, the distance between the two trajectories gets so large that the trajectories can’t get any farther apart. After that, their motion appears to be completely independent as they wander around chaotically. This is what we mean by sensitive dependence on initial conditions: a small initial difference can cause a large difference in the motion at later times.
Definition of the Lyapunov Exponent
We can describe this more precisely. Choose two nearby initial locations. We will call the location of the blue dot $x$ and the location of the red dot $y$. Both of them are functions of time $t$. Let $x(t=0)$ be the first initial location. Its trajectory is $x(t)$: the position at any time. We could choose another nearby initial location $y(0)$. The trajectory starting from this initial location is $y(t)$. Call the difference between the two initial locations to be $\delta(0) = y(0) – x(0)$. Similarly, $\delta(t) = y(t) – x(t)$. The distance between the two trajectories is the absolute value of the difference: $|\delta(t)|$.
We said above that the distance between the two trajectories grows exponentially. We can write this as: $$ |\delta(t)| \sim e^{\lambda t} |\delta(0)| \, . $$ $\lambda$ is the exponential growth rate and is called the Lyapunov exponent. This is only an approximate equality. It is not easy to figure out all of the details of how the trajectories get farther apart. We just know that this growth tends to be exponential.
Solving for $\lambda$ gives: $$ \lambda \sim \frac{1}{t} \log \frac{|\delta(t)|}{|\delta(0)|} $$
It would be nice if we could replace the approximate sign with an equal sign.
We don’t want to worry about all of the details of the motion: we just want to look at the average way that the two trajectories get farther apart. One way of doing this is by looking at longer and longer times. If you only look at a short time, the details of the motion become important. If you look for a long time, all of the details blur together and average out. Instead of calculating $\lambda$ for some short amount of time, take the limit as $t \rightarrow \infty$.
This introduces another problem. The two trajectories don’t get farther and farther apart forever. After a long time, the two trajectories are moving independently, and they aren’t getting farther apart. In the example of the logistic map, the maximum value of $\delta(t)$ was one. The ratio $|\delta(t)|/|\delta(0)|$ is at most one for any time, while $1/t$ approaches zero. The result of this limit would always be zero.
How long does it take for the distance between the trajectories to get so far apart that the motion looks independent? The answer depends on how close the two initial locations are to begin with. The closer the two initial conditions are, the longer the resulting trajectories will stay together. This gives us a way to deal with the problem raised in the previous paragraph. Instead of just taking the limit as $t \rightarrow \infty$, we take two limits simultaneously: the limit as $t \rightarrow \infty$ and the limit as the two initial locations get closer and closer together, i.e. $\delta(0) \rightarrow 0$.
The complete definition of the Lyapunov exponent is: $$ \lambda = \lim\limits_{\delta(0) \rightarrow 0 \ , \ t \rightarrow \infty} \ \frac{1}{t} \log \frac{|\delta(t)|}{|\delta(0)|} $$
If you aren’t familiar with limits, you can use the previous definition. It is then an approximate relationship, which works when the two initial conditions are close together and after you’ve looked at the trajectories for a long time, but not too long of a time.
Lyapunov Exponents for Chaos
We can calculate the Lyapunov exponent for any motion. If the motion is chaotic, the Lyapunov exponent will be positive. If the motion is not chaotic, we will calculate the Lyapunov exponent to be zero, or even negative.
An example of motion that is not chaotic is moving around a circle at constant speed. If the motion begins with two different starting positions, the distance between the two trajectories will not change. $\delta(t)$ is always equal to $\delta(0)$, so their ratio is one. We can calculate the Lyapunov exponent: $$ \lambda = \lim\limits_{\delta(0) \rightarrow 0 \ , \ t \rightarrow \infty} \ \frac{1}{t} \log(1) $$ This is automatically zero because $\log(1) = 0$. Even if the $\log$ of the ratio were nonzero but didn’t grow in time, then the Lyapunov exponent would still be zero because $1/t$ gets smaller and smaller as $t \rightarrow \infty$. Most of the time you calculate a Lyapunov exponent, it will be zero.
Chaotic systems have positive Lyapunov exponents. This is equivalent to saying that they have sensitive dependence on initial conditions.
You can have a negative Lyapunov exponent if everything is getting exponentially closer and closer together.
Let’s numerically calculate the Lyapunov exponent for the logistic map with $r = 4$. Limits are hard to take numerically, but we will do our best. Start with an initial condition $x(0) = 0.1$ and initial difference $\delta(0) = 10^{-10}$. Plot $\log(|\delta(t)| / |\delta(0)|)$ vs time. We expect that the plot will show a straight line. The slope of this line is the Lyapunov exponent.
The plot has two distinct regions. For $t < 40$, the data are increasing approximately linearly. This is the region where we measure the Lyapunov exponent. The slope of the red best fit line for this region is $\lambda = 0.63$.
For $t > 40$, the data are approximately horizontal. After this point, the distance between the two trajectories is close to one.[2]On the left axis, plug in $|\delta(0) = 10^{-10}|$ and $|\delta(t)| \approx 1$. When the error has grown from $10^{-10}$ to $1$, the left axis goes from $0$ to $\log(1/10^{-10}) \approx 23$. The trajectories are both trapped in the region between 0 and 1, so they can’t get any farther apart. This is the region where the two chaotic motions wander around independently.
Along with these trends, the motion has some smaller wiggles. These are the details of the motion that we wanted to average over by taking the limit as $t \rightarrow \infty$.
We can see that the logistic map with $r = 4$ has a Lyapunov exponent of about $\lambda = 0.63$. This motion is chaotic, so its Lyapunov exponent is positive.
The Lyapunov exponent could be more difficult to measure in a more complicated chaotic system. In the logistic map, $x(0)$ was just a number. If the object could move around in three-dimensional space, then $x(0)$ would be a vector. If we wanted to describe how a fluid moves, then the initial condition would have to include the density at every place that the fluid is located. $x(0)$ in this case would be a more complicated mathematical object know as a field, which can contains even more information. The definition of the Lyapunov exponent requires you to be able to measure distance. It is easy to figure out how to measure the distance between two locations. It much harder to quantify the difference between two moving fluids. This can be done, but you have to be careful here.
I will add another word of caution about Lyapunov exponents. In some physical situations, an object can escape and never return. An example is two drops of water which start on either side of a continental divide. They may begin very close together, but as they flow downhill, they get farther and farther apart. Similarly, in some mathematical models, a trajectory will extend out to infinity. If either of these happens, then you can get a positive Lyapunov exponent without having chaos. The trajectories are moving exponentially farther apart, but this behavior is simple. The Lyapunov exponent, by itself, cannot be used to define chaos.
Despite these complications, the essence of sensitive dependence on initial conditions remains the same. The motion starting from two similar initial conditions rapidly becomes dramatically different.
A Physical Example: Pinball
We now turn to a physical example of chaos, the game of pinball. The game relies on the competition between the unpredictable motion of the ball and the reactions of the player.
The Lyapunov exponent for a simplified game of pinball can be calculated by hand. This does involve more arithmetic (especially trigonometry) than I usually include, so I’ve relegated the details to Appendix A. I’ll include a simple argument here that allows us to guess most of the right answer.
In many games of pinball, there are three circular disks near the top of the board.
We will consider an idealized game of pinball focused on these three disks. The ball travels on a perfectly smooth, frictionless surface. The disks are perfectly circular. When the ball bounces off a disk, it neither gains nor loses energy. In an actual pinball game, the ball gains energy when it hits the disks. This energy compensates for the energy lost to friction.
It is a typical trick of physicists to make a series of simplifications to turn a complicated problem into a solvable problem and hope that the solution is close enough. Once we’ve solved the simple problem, we might then try to add the complications back in as a small modification of the simple problem. I’ve oversimplified here and I won’t add the complications back in later, so I should explain why I think my conclusion will still hold.
We are trying to measure how quickly small initial differences grow. Any further complications will add more random or complex behavior, making the initial differences grow faster. Our estimate for the Lyapunov exponent will be smaller than the actual Lyapunov exponent. If the calculation gives a positive Lyapunov exponent, the actual one must be positive as well.
Our goal is not to explicitly determine the trajectory of two initial ball locations as they bounce among the three disks. Our goal is to determine how the difference between two trajectories changes. At each bounce, we have a new $\delta$. We want to determine how each subsequent $\delta$ depends on the previous $\delta$. A typical pair of trajectories is shown in Figure 4.
How does $\delta_1$ depend on $\delta_0$ and the geometry of the disks?
If the initial difference, $\delta_0$, were larger, the difference after one bounce, $\delta_1$ would be larger too.
If the radii of the disks were smaller, then the pinball would be bouncing off something with larger curvature. The angle between the two trajectories after the bounce would be larger. This trend can be most easily seen by considering limiting cases. If the pinballs bounced off a flat wall, the trajectories would leave the wall with the same angle. If there were a corner in the wall, the trajectories could come off at totally different angles. When the disks are smaller, they are more curved, and he angle between the trajectories is larger. If $R$ were smaller, $\delta_1$ would be larger and if $R$ were larger, $\delta_1$ would be smaller.
If the distance between the disks is larger, the trajectories spread out more after striking the first disk. The pinballs are no longer moving in the same direction, so they become farther and farther apart. If $L$ were larger, $\delta_1$ would be larger.
So $\delta_1$ should be directly proportional to $\delta_0$ and $L$ and inversely proportional to $R$: $$ \delta_1 \sim \frac{L}{R} \, \delta_0 $$
If we do the calculation (see Appendix A), we would see that there is an extra factor of two. The actual relationship between the difference between the trajectories before and after the bounce is $$ |\delta_1| \geq \frac{2 L}{R} \ |\delta_0| \, . $$
This relationship doesn’t just hold for the first two bounces. It holds for any two bounces. We can apply this equation over and over again to determine what happens after a larger number of bounces, which we will refer to as $b$. $$ |\delta_b| \ \geq \ \frac{2 L}{R} \ |\delta_{b-1}| \ \geq \ \left(\frac{2 L}{R}\right)^2 |\delta_{b-2}| \ \geq \ \dots \ \geq \ \left(\frac{2 L}{R}\right)^b |\delta_0| \ . $$
In this problem, time is measured by the number of bounces. $b$ appears in the exponent. The distance between the two trajectories grows exponentially. The pinball machine has a positive Lyapunov exponent:[3]We also need to know that $2L > R$, but if this were not true, the disks would overlap. $$ \lambda \geq \log\left( \frac{2 L}{R} \right) $$
Let’s plug in some numbers to see just how quickly the difference between the two trajectories grows.
A typical pinball table might have disks with radius of about one inch (2.5 cm). The disks are about four inches (10 cm) apart. The Lyapunov exponent in this case is $e^\lambda \geq 8$.
Suppose that the original distance between the two trajectories was the diameter of an atom, about one angstrom or $10^{-10}$ meter: $\delta_0 = 10^{-10}$ meter. After ten bounces, the distance between the two trajectories will be $$ \delta_{10} \geq 8^{10} \ \delta_0 \approx 11 \ cm \, . $$
This is bigger than the distance between the disks. If one of the trajectories continues bouncing among the three disks, the other trajectory will miss the disks entirely. After this, it doesn’t make sense to use this calculation anymore because at least one of the pinballs has bounced away from the three disks and escaped to another part of the pinball machine.
Indeterminism
That last result is astonishing.
Two pinball games which start out only an atom width’s apart become completely different from each other in ten bounces.
Pinball is essentially the same game as billiards: both games involve trying to predict how hard balls will bounce off of various objects. Billiards is a classic example of determinism. A skilled player who knows her geometry and has precise control over her hits can predict how every shot will pan out. The same cannot be said of pinball.
The sensitive dependence on initial conditions characteristic of chaos challenges traditional notions of determinism.
Determinism is the idea that the universe runs like a clock. Once the clock is wound up (once the initial conditions are determined), then the rest of the behavior can, in principle, be exactly determined using the laws of physics. Laplace gives perhaps the best description in A Philosophical Essay on Probabilities (1814):
We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.
Our pinball example doesn’t immediately contradict determinism in principle. If you know the exact starting location and velocity of the pinball, then you could exactly determine its future motion. It does, however, give an idea of how difficult this would be.
All experiments have finite precision. Every measuring device has a smallest tick mark and every electronic readout has a finite number of decimal places. The ‘intellect’ which can exactly measure initial conditions described by Laplace is impossible for us to achieve. For us, the more relevant question is approximate determinism, not exact determinism. If we can measure the initial conditions approximately, can we make approximate predictions?
If we know the precision with which we can measure the initial conditions, we can use the Lyapunov exponent to estimate how long we can make good predictions for.
To predict just ten bounces of a game of pinball, you would need to know the starting location with precision better than the diameter of an atom.
Measuring the location of a macroscopic object with precision of the diameter of an atom is impossible in practice. Even if this were technically feasible, higher precision also becomes impossible in principle. Heisenberg’s uncertainty principle (one of the fundamental laws of quantum mechanics) puts a restriction on how precisely anyone can measure the position and momentum of any object. For macroscopic objects, this is usually irrelevant because the precision accessible with our best instruments is much worse than the precision limits imposed by the uncertainty principle. The uncertainty principle becomes relevant once we consider distances about the size of an atom or smaller.
As long as the classical mechanics of Newton, Lagrange, and Hamilton reigned supreme, the clockwork universe seemed plausible. It has suffered many challenges in the centuries since it was introduced. The first challenge was the irreversibility of the second law of thermodynamics, which states that entropy always increases. The response to this challenge was to claim (without proof) that each atom moves deterministically and that thermodynamics only looks the statistics of large numbers of atoms. It is not surprising that statistical approximations, even good ones, are not deterministic. However, in the early 1900s, we discovered that the motion of atoms is not deterministic and has to be described by quantum mechanics.
Chaos makes determinism untenable even in its original setting – macroscopic objects moving according to Newton’s laws. The pinball machine is classical, just like the deterministic game of billiards. However, predicting the future behavior of the pinball machine requires so much precision that the indeterminism of quantum mechanics sneaks back in.
Chaos does not directly contradict exact determinism. It does contradict approximate determinism and allows quantum indeterminism to rapidly become macroscopic.
Appendix A: Pinball Calculation
Problem Statement
The goal of this calculation is to figure out how the distance between two nearby trajectories changes as they bounce around the top three disks. The strategy we will use to solve this problem has two parts: (1) determine how different initial locations of a bounce change the angle the pinball is reflected towards and (2) use the difference in angle to determine how far apart the pinballs will be before the next bounce.
Two pinballs approach the disk. This has to happen at different times so the pinballs don’t collide. To make this calculation simpler, assume that the two pinballs move in the same direction, but start at slightly different locations. Draw a line through the center of the disk that is parallel to the direction of each pinball’s motion. Use $x$ to refer to the distance between the first pinball’s incoming trajectory and the line through the center of the disk and $y$ to refer to the distance between the second pinball’s incoming trajectory and the line through the center of the disk.
Both trajectories strike the disk – the first at angle $\theta$ and the second at angle $\phi$. These angles are related to $x$ and $y$ using SOH-CAH-TOA, with $R$, the radius of the disk, as the hypotenuse of the right triangle. $$ x = R \sin\theta \ , \ y = R \sin\phi $$ These angles are also the angles of incidence for the two trajectories. The angle of incidence is the angle between the incoming trajectory when it touches the disk and the line out from the center. This can be simply seen by sliding the angle $\theta$ (or $\phi$) along the hypotenuse of the right triangle used above until it reaches the circumference of the circle.
Determining the difference between the two angles
There is one relevant physical law. When any object bounces off of a hard surface, the angle of incidence equals the angle of reflection. The angle at which the object approaches the surface is the same as the angle at which the object leaves the surface. This explains why $\theta$ and $\phi$ also appear on the diagram immediately after the bounce.
Define $\psi$ to be the angle between the two reflected trajectories. From the diagram, we can see that $$ \psi = 2 \phi \, – 2 \theta \, . $$ The larger red angle is $2\phi$, the smaller red angle is $2\theta$ and $\psi$ is the difference between these two. We will use $\psi$ to relate the difference $\delta_0 = y – x$ to $\delta_1$.
As long as $\delta_0$ is small, then $\epsilon = \phi – \theta$ will also be small. We will use the small angle approximation: $$ \sin\epsilon \approx \epsilon \ \mbox{and} \ \cos\epsilon \approx 1 \ \mbox{when} \ \epsilon \ll 1$$
Also recall the trigonometric identity: $$ \sin(\theta + \epsilon) = \sin\theta \cos\epsilon + \sin\epsilon \cos\theta $$
Now it’s time for some calculations.
We can replace $\phi$ with $\theta + \epsilon$, where $\epsilon$ is the small difference between the angles. $$ \begin{array}{rcl} \delta_0 & = & y-x \\ & = & R \sin\phi – R \sin\theta \\ & = & R \sin(\theta + \epsilon) – R \sin\theta \\ & = & R \sin\theta \cos\epsilon + R \sin\epsilon \cos\theta – R \sin\theta \\ & \approx & R \sin\theta + R \ \epsilon \cos\theta – R \sin\theta \\ & = & R \ \epsilon \cos\theta \end{array} $$
We also know from the definitions of $\psi$ and $\epsilon$ that $$ \psi = 2 \epsilon \, . $$ We can use this to relate $\psi$ and $\delta_0$. $$ \psi = \frac{2 \delta_0}{R \cos\theta} $$
Determining how much the pinballs separate before the next collision
After bouncing off the first disk, the two trajectories will travel a distance $L$ before reaching the next disk.
Since they are moving at different angles, they will get farther apart as they travel. The distance between them, which we call $\delta_1$, is approximately the length of the arc with angle $\psi$ and radius $L$. $$ \delta_1 = L \psi = \frac{2 L}{R} \sec\theta \ \delta_0 $$
We know that the magnitude $\cos$ is always less than or equal to one, $ -1 \leq \cos\theta \leq 1 $, so the absolute value of $\sec$ is always greater or equal to one, $ |\sec\theta| = |1 / \cos\theta| \geq 1 $. By the time the trajectories reach the next disk, the distance between them is at least $$ |\delta_1| \geq \frac{2 L}{R} \ |\delta_0| \, . $$
This is the solution we were looking for.
References
| ↑1 | I will describe topological entropy in Part VII. |
|---|---|
| ↑2 | On the left axis, plug in $|\delta(0) = 10^{-10}|$ and $|\delta(t)| \approx 1$. When the error has grown from $10^{-10}$ to $1$, the left axis goes from $0$ to $\log(1/10^{-10}) \approx 23$. |
| ↑3 | We also need to know that $2L > R$, but if this were not true, the disks would overlap. |