I chose the name The Chaostician for this blog. To live up to this name, this blog should be about chaos, in particular, what a physicist means when referring to ‘chaos theory’.

This eight part series is my explanation of chaos theory to a popular audience. Chaos is a mechanism that allows deterministic objects to behave unpredictably. I will explain why this happens and what kinds of predictions we can make when something is chaotic.

Those of you who are already experts in nonlinear dynamics know that there are many ways to approach chaos. In this series, I will focus on behavior on strange attractors in dissipative chaos. In particular, the goal is to explain the ideas underlying periodic orbit theory and trace formulas you can use to calculate expectation values. Other series will address other approaches to chaos.

This is the final post for this series. When the motion is chaotic, it is impossible to make long time predictions for a particular trajectory, but it is possible to make long time statistical predictions. I hope to explain the basic ideas of how we can calculate these long time averages for a strange attractor.

---

Part I: Introduction.
Part II: The Simplest Chaotic System.
Part III: Lyapunov Exponents.
Part IV: Strange Attractors.
Part V: Continuous Time.
Part VI: Stretch and Fold.
Part VII: Partitions and Symbols.
Part VIII: Periodic Orbit Theory.

---

Prerequisites: I hope this is understandable using high school level math, until the Appendix, which has much harder math. If you don’t understand anything, please ask – in the comments or by emailing thechaostician@gmail.com.

Originally Written: August – November 2017.

Confidence Level: Established science since the mid twentieth century. Occasional philosophy mixed in.

---

---

Unstable Periodic Orbits in the Strange Attractor

Although almost all orbits on the strange attractor are chaotic, if you are willing to looking for needles in a haystack, you can find isolated spots of regularity.

As the chaotic motion whirls around the attractor, it occasionally returns to a location close to where it was before. Whenever this happens, you can look for a periodic orbit nearby.

If you start at exactly the right location, you will return to the same location that you started. If you start even slightly off, each time you go around, you get farther and farther away from where you started until you chaotically wander about the strange attractor like any other orbit. This is what we mean when we say that the periodic orbit is unstable.

Since the motion is periodic, the symbolic dynamics will also be periodic. Instead of writing out the entire symbolic dynamics, we can just write the symbols until they begin to repeat. For example, the Rössler system has a periodic orbit with period 5.89 with symbolic dynamics: UUUUUUUUUUUU… , but we refer to it just as U. It also has a periodic orbit with period 11.77 with symbolic dynamics: LULULULULULU… , but we refer to it just as LU. As a reminder, the period is the length of time it takes to return to its initial position.

Some of the shorter unstable periodic orbits for the Rössler system are shown below. A typical trajectory on the attractor is shown in red and the periodic orbit is shown in blue. I encourage you to trace them out and see that the orbit does have this symbolic dynamics. Since you can start at any point along the orbit, you can shift the symbols without changing the orbit: LU is the same as UL and LUULU is the same as LULUU.

Along with the periodic orbit with period 5.89 with symbolic dynamics U, the techniques for finding periodic orbits will also find one with period 11.78 and symbolic dynamics UU and one with period 17.67 with symbolic dynamics UUU. These are obviously not new periodic orbits – they are just repeats of the 5.89 periodic orbit. We should exclude these repeats and only keep the ‘prime’ periodic orbits.^[1]The analogy here is with prime numbers. Just like the number four is two, repeated twice, the orbit with period 11.78 is the orbit with period 5.89, repeated twice.

Notice that we can find more periodic orbits as the period becomes larger. This trend continues: there are infinitely many periodic orbits with arbitrarily long periods. Any symbolic sequence that follows all of the pruning rules can be repeated. There exists a periodic orbit which corresponds to this symbolic sequence. If we chose a good partition, there will be only one periodic orbit for each symbolic sequence.

The periodic orbits shown here have at most six symbols before they repeat; I have purposefully excluded longer periodic orbits. Are we missing any short periodic orbits?^[2]Answer: We are missing LUUUUU. This symbolic sequence follows the pruning rules, so there should be a periodic orbit corresponding to it.

---

Shadowing

Periodic orbits are valuable because they connect short and long times.

To find a periodic orbit or to check that you have one, you only need to compute the motion of the chaotic system for one period. The period of the periodic orbit that you are interested in is (hopefully) short enough for this computation to be meaningful. If you compute for too long of times, the positive Lyapunov exponent will amplify the numerical errors in the computation and render your result meaningless.

Once we have found a periodic orbit, we know that it exists for all time. If you start exactly on the periodic orbit, you will stay on it forever.

We can use periodic orbits to connect the long time behavior of the chaotic system to short time computations.

To make this connection, we need a way to think of generic long time chaotic motion in terms of short periodic orbits. We call this connection ‘shadowing’.

For many strange attractors, periodic orbits are dense on the attractor. Regardless of where you are on the attractor, there is a periodic orbit nearby. All of the examples we’ve looked at have this property.^[3]The most common way for this to fail is if your equations of motion have some kind of continuous symmetry. In this case, you replace periodic orbits with quasiperiodic orbits or some other invariant … Continue reading

A generic chaotic orbit starts out close to one periodic orbit. It follows it for a while, but also starts to drift away, because of the positive Lyapunov exponent. When it drifts away, it comes close to another periodic orbit and follows that one for a while before drifting away again.

We can even see this by looking at the periodic orbits themselves: periodic orbits with a long sequence of U’s in the Rössler system tend to go close to the shortest periodic orbit U for several times around before drifting away.

A generic chaotic orbit can be thought of as a sequence of which periodic orbits it visits.

The mathematical statement of this is called the ‘shadowing lemma’. If we can construct an approximate orbit, then, with a few assumptions, there must be an actual orbit nearby. The approximate orbit is the sequence of periodic orbits, each of which can be computed in a short amount of time. There must then be an actual orbit that stays close to this sequence. The actual orbit cannot be computed directly – it exits for a long time, so any numerical errors will be dramatically amplified by the positive Lyapunov exponent – but we know that such an orbit exists.

This is Poincaré’s strategy for thinking about dynamics. First, find periodic orbits or other invariant objects.^[4]A periodic orbit is an invariant object because, if you start on it, you stay on it forever. If you just drew some random circle, that would not be true. A strange attractor is also an invariant … Continue reading Use these objects as a skeleton for the dynamics. The generic behavior can be understood using the geometry of these objects and the motion along them.

---

Statistical Predictions

Now we are finally in a position where we can think about how to make statistical predictions.

Although most trajectories are unpredictable, they shadow the predictable paths of periodic orbits. We will use these periodic orbits for our predictions.

What all will appear in these predictions?

Basic information about the periodic orbits is obviously important. We will need to know the geometry and period of each periodic orbit.

As long as there is only a single basin of attraction, after a long time, a generic orbit will have come close to every periodic orbit. Our statistical predictions will involve a sum over all (prime) periodic orbits.^[5]Since there are typically infinitely many periodic orbits in a chaotic system, we usually can’t sum over all of them. Instead, we take only a finite number of terms in the sum. This sum … Continue reading

The last piece of information that we need is how long the generic chaotic orbit stays close to each periodic orbit. This depends on how quickly nearby orbits approach or leave the neighborhood of the periodic orbit. The Lyapunov exponent gives some sense of this, but we need more information. For each periodic orbit, we need to calculate the rates at which orbits in a neighborhood expand or contract. The more rapidly the neighborhood of the periodic orbit expands, the less time a generic periodic orbit will spend near it, and the less important that neighborhood is for the sum.

If you want to find the average position (or standard deviation of the position, or average speed, or any other long time average) of a chaotic orbit, you need to sum over the average position of each periodic orbit, weighted by how long the orbit is and how much time a typical orbit spends near it. The mathematical details are a mess, so I will relegate them to an appendix. In order to make a statistical prediction about the long-time behavior of something chaotic, you only need to know a few things about the unstable periodic orbits embedded in the chaos.

---

Conclusion

Chaos is not the opposite of order. Chaos is the mechanism by which complex, unpredictable behavior arises from simple, deterministic rules. Structure can be found within this complexity. The structure, whether it is a period doubling cascade, a strange attractor, or a homoclinic tangle, shows surprising universality.^[6]I haven’t described homoclinic tangles in this series, but I intend to at some point. A particular structure can be found in many, extremely different physical systems, but the structure has some precise properties that are the same in all of these different settings.

Something is chaotic if it has positive topological entropy, i.e. if the number of distinct patterns in which it can move increases exponentially in time. In order to have this exponential growth, two things must occur.

First, stretching. Two locations which are initially near each other must get farther and farther apart. The simplest way to quantify this sensitive dependence on initial conditions is with the Lyapunov exponent. In order to become distinct, orbits must get far enough apart to enter different regions of the partition. In order for the number of distinct trajectories to grow exponentially (i.e. there is a positive topological entropy), the distance between nearby points should grow exponentially (i.e. there is a positive Lyapunov exponent).

Second, folding. If nearby points get farther apart and there is no way to bring them back together, they will expand forever. Folding causes the motion to come back close to its original location again. The shadowing lemma tells us that there should be a periodic orbit somewhere nearby. When these periodic orbits exist, then we can use periodic orbit theory to calculate the statistical properties of the long-time behavior.

If the motion both has sensitive dependence on initial conditions and returns to close to its initial location, then it is chaotic. These features can arise either in discrete time or in continuous time. They can arise either for finite collections of objects or for continuous descriptions of matter (like fluids). They can arise in almost any physical setting. Once you notice this structure, you can use the tools of chaos theory to describe and understand the dynamics.

---

---

Appendix: Trace Formula

Prerequisites: Significantly more math than I usually include in a blog post. In particular, you need to know about the Dirac delta function, operators (like a matrix, but for functions instead of vectors), Jacobian matrix, and maybe also some statistical mechanics. It is very hard to make what I’ve described precise.

---

Instead of thinking about an individual trajectory, it helps to think about how collections of trajectories move. We’ve one such collection already: the global stationary distribution, but you could take any collection you want. We can then move all of the trajectories forward in time together. Call this process $\mathcal{L}^t(y,x)$, an operator which moves trajectories starting at points $x$ to points $y$ after time $t$. If each trajectory moves according to $y = f^t(x)$, then the operator can be written using a Dirac delta function: $$ \mathcal{L}^t(y,x) = \delta(y – f^t(x)) $$

The trace of $\mathcal{L}$ only involves the points for which $y = x$. These are the points which return to their starting point after time $t$. The trace of the operator only depends on the periodic orbits: $$ \mbox{tr} \ \mathcal{L}^t = \sum_p’ T_p \sum_{r=1}^\infty \frac{\delta(t – r T_p)}{| \mbox{det} ({\bf 1} – M_p^r) |} $$ The first sum is the sum over prime periodic orbits, which is why the sum is marked with a prime. Each prime periodic orbit is labeled by $p$, which tells its symbolic dynamics, and has period $T_p$. For the Rössler system, this sum would be U + LU + LLU + LUU + … . The second sum is the sum over how many times around each prime periodic orbit you’ve gone. $r$ counts how many times around you’ve been and ranges from 1 to $\infty$. The top of the fraction is a Dirac delta function which ensures that the time matches the period of this (prime or unprime) periodic orbit. The bottom of the fraction is a measurement of how long a typical orbit spends near this periodic orbit.

$M_p^r$ is the monodromy matrix for the prime periodic orbit $p$. The monodromy matrix is the Jacobian matrix perpendicular to the direction of motion. Linearize the equations of motion in a small neighborhood near the periodic orbit. Solve this linear system with time dependent coefficients for one period. The result is the monodromy matrix multiplied by the vector of initial conditions. To go around the prime periodic orbit multiple times, raise the prime monodromy matrix to the power $r$.

This trace is not very well behaved mathematically. It is zero for all times except the periods of periodic orbits. To smooth it out, we take a Laplace transform (replacing $t$ with $s$). We can then exponentiate it to turn the trace into a determinant using $\mbox{det} (e^\mathcal{A}) = e^{\mbox{tr} \mathcal{A}} $. It is easier to use $\mathcal{A}$, which is related to $\mathcal{L}^t$ by $\mathcal{L}^t = e^{t \mathcal{A}} $. The result is the ‘spectral determinant’: $$ \mbox{det} (s – \mathcal{A}) = \exp \left( – \sum_p’ \sum_{r=0}^\infty \frac{1}{r} \frac{e^{-s T_p r}}{| \mbox{det} ({\bf 1} – M_p^r) |} \right) $$ This is a spectral determinant because the left hand side allows us to calculate the eigenvalues of the operator, also called its spectrum.

The spectral determinant is used like the partition function of statistical mechanics. In statistical mechanics, the goal is the sum over many atoms to calculate macroscopic quantities like pressure or temperature. This sum is hard to do, so we don’t want to have to do it for every macroscopic quantity. Instead, we do the sum once to find the partition function and then use the partition function to calculate the pressure, temperature, or whatever else we want.

Instead of summing over atoms, we are summing over periodic orbits. This is hard enough that we only want to do it once, to find the spectral determinant. Once we have the spectral determinant, we can use it and its derivatives to calculate any other statistical quantity we want.

When we actually do the sum, we can’t add over all of the periodic orbits because there are infinitely many of them. Instead, we sum over the short periodic orbits that we found. This introduces some error. The error is proportional to $e^{-T_\star}$, where $T_\star$ is the period of the shortest (prime) periodic orbit that you did not include in the sum.

For more details about how all of this works, there is an entire textbook available online, called chaosbook.