Welcome back to chaos. Lets remind ourselves where we are in our detective story. The mystery started with Newton's clockwork universe, which seemed to rule out the possibility of chaos once and for all. It's deterministic character seemed to leave out all room for chance. No possibility for chaos.

Yet then came the first clue, that something was rotten in the clockwork! Remember that this was the intractability of the three-body problem in astronomy. Hundreds of years go by and no one can solve it.

Next comes what seems like a break in the case. Based on visualization, the new technique of Poincare, the famous French mathematician, thinks he may have a way of solving the three-body problem. In the course of his work, he thinks at first he's solved it, enters the contest, submits his brilliant solution, and wins the prize. Yet a question comes up about his method, so that he then realizes that he made an error. He catches his error and works feverishly to figure out what's going on, and then in the course of that, it turns out he has not solved the problem, but has discovered something much greater, deterministic chaos.

The phenomenon being that simple deterministic systems which have no element of chance in them, nothing random about them, can look like they're behaving randomly and can act unpredictable, wildly so. Poincare is not happy about it and recoils from the implication, but nobody else is really worried because they don't understand what he did. It's so arcane and abstract, involving thinking in 18 dimensional space, which Poincare can do, but nobody else can.

So that's where things sit, and what brings us to our lecture today. So for the next 70 years after Poincare, chaos remains a really pretty quiet backwater of science. Not the hottest area in town, certainly not. It's cultivated only by a small priesthood really, a mathematical priesthood that's devoted to thinking about what Poincare did. So this is on the order of maybe 5-10 people in the world.

To understand why, this is an era when science was becoming increasingly specialized, so that math itself was starting to look inward. That happens once in a while in the history of science, when scientists feels expansive and they are talking to each other. Yet other times, they become introspective and concerned with only their own subject.

For example, in the 1700s and 1800s, the era of Newton and afterward, people were in the expansive mood. The best mathematicians also did astronomy and physics. Yet now, just beginning the turn of the 20th century, they only really wanted to think about math itself. Why was that? Through all their interactions with physics and astronomy, they had started to think intuitively and gotten into certain logical paradoxes, it turned out, so that there were certain things you'd want to do by common sense, which turned out to have mathematical problems. So there came to be a feeling that we needed to be more rigorous, and math kind of corrected its own foundations, and turned inward at that point, focused on its own internal structure.

Physicists meanwhile had their own problems. Now it was the era of atomic physics. People were worrying about electrons and radiation, only recently discovered. Radioactivity, things like that. So they were in no mood to be fretting about a problem (the three-body problem) in classical Newtonian physics. That was done, as far as everyone was concerned, and the action was somewhere else.

So this was then an era when there really was no great motivation to think about what Poincare had done. his flame almost went out, yet it was kept barely flickering by an American mathematician name George David Birkhoff 1884-1944, who taught at Harvard and was himself a very fine mathematician, notable for being the first to e trained completely in America. That was unusual since the best people went to Europe for their training, yet not Birkhoff. He stayed here, taught here, and trained a number of other American mathematicians who followed him.

Along with Birkhoff there was also work eing on dynamical systems, which is what Poincare's area had come to be called, by a handful of mathematicians in the Soviet Union, and in Europe, and England.

Now curiously, what really kept Poincare's work going, was not these mathematicians. Although they contributed to it quite a bit, the engineers really stepped in. We haven't talked much about engineering in this course, but there were mathematically inclined engineers, lets say, as many of them are, who were interested in what Poincare had done, but not with regard to chaos.

That was an interesting thing, since Poincare's work was so powerful that it touched on all kinds of complicated problems related to differential equations, and these engineers wanted to use his methods to analyze what were called oscillators. These were devices that could produce clean, periodic signals, for example like when thinking about the electrical oscillations of vacuum tubes, which were only recently developed. These were new fangled technologies being used in the first radios, telephones, and later in radar, television, and even lasers.

So oscillators became a big topic in electrical engineering, and while studying them, these mathematically minded engineers found it useful to extend some of what Poincare had done, as far as his visual approach. Yet in the course of their experiments, what sometimes would happen is that these engineers would trip across their own hints of chaos. For example there was a team of Dutch engineers who were noticing a peculiar hiss in a circuit they were studying. It didn't seem to them like the usual random noise that sometimes afflicted radio circuits, but they didn't know what it was. They reported it, but couldn't really make sense of it, and so they then just left it, and that was that.

In retrospect it was clear that they were hearing a sound of chaos, deterministic chaos in their circuit, which was later found out be British mathematicians, Mary Cartwright and J.E. Littlewood. Likewise, a Japanese researcher named Chihiro Hayashi, spent a longtime looking at oscillators, and had observed chaos dozens of times. Yet he never reported it. Why not, and how do we know he was even thinking about it?

Because he had a student, as we fast forward years later to 1970 or 80 when chaos was in full flower, who reports the first electrical chaos in circuits. His adviser, the famous Hayashi, says that he has seen this dozens of times, going back many years into the 60s. His student Yoshisuke Ueda asks why he never told him about it or published it. This was indeed the case, since Hayashi had simply kept it to himself, and didn't know what he even had. He may have thought his circuits were malfunctioning.

The point here is that without any conceptual framework for thinking about chaos, either because people were unaware of Poincare, or unable to understand him, or just didn't think his work applied in their context, researchers dismissed the case of chaos they were observing directly. They could have just ignored it.

Yet what Steven thinks was really happening in many cases was that they simply couldn't see what they were seeing. It's a curious thing about human psychology, that if you don't have the right mental framework, you sometimes can't see what's right in front of your face.

Well lets now come to the main topic of this lecture, which is the first time after this calm period after Poincare ends. The calm ended with a thunderclap, which seems appropriate, since the man who created it was one obsessed with weather prediction and storms. So in the rest of this lecture, we'll hear the dramatic story of the work of Edward Lorenz 1917-2008, a meteorologist at MIT. His rediscovery of chaos was in a much more dramatic and startling form that what Poincare had seen. One really can't miss it now, you would think, after Lorenz' work.

Lorenz found this full blown chaos in a model of weather patterns that he developed. Before we hear about this fantastic work, we have to hear about this man. Steven had the privilege of getting to know him as a young assistant professor working at MIT, and he himself was a meteorologist working there in the earth and planetary sciences department.

Now everybody regards professor Lorenz as one of the great pioneers of chaos theory. He is universally admired, and respected. So it's particularly striking when you meet him, that this is an absolutely modest and unassuming man, who does not come across as brilliant. He really reminds one of some kind of roadside farmer you may meet when going out to buy vegetables or something. He really seems like a Yankee farmer, like the guy who does the Pepperidge Farm commercials!

So every year they would go through a certain little ritual that Steven came to have an affectionate feeling of anticipation for. Steven was teaching a class on chaos, and felt he had to invite professor Lorenz to come talk the students, after spending three weeks on the Lorenz equations, which we'll hear about in this lecture and the next three to come. So he had students with all this backgrounds in the Lorenz equations, and professor Lorenz is about three minutes away in the next building, so Steven would have him come to the class.

He would call him up, and say, "Dear professor Lorenz, would you be willing to come talk to my class?" He would reply in his monotone, "Yes, I would be happy to come." So Steven would say that was fantastic, but then the professor would always ask, maybe six years in a row, "What would you like me to talk about?"

Sowhat should you talk about professor Lorenz? how about the Lorenz equations? It's like asking Einstein to talk about relativity to the class, how about the Lorenz equations? Yet then he would always say, "That little model?" In other words, you really want me to talk about that little thing? Then he would come to the class, and not talk about it! At this point he was 70 or 80 years old, and he would talk about whatever he was working on at the moment. He was still probably publishing.

Yet the class never cared, because they were so awestruck and thrilled to be getting a glimpse of this man who created modern chaos theory. And in fact, whatever he was doing at the time, was pretty interesting too. So this little model, is what we'll be hearing about, and it really did change the way we look at the world.

Now Lorenz himself was born in 1917 and grew up in West Hartford, Connecticut. As a boy, he liked weather, and was certainly interested in it, yet he wasn't what one would call a weather bug. What he really loved was math! He went to college at Dartmouth, studied math and thought he would go into it. By 1938 he got his undergraduate degree in math. Then he went on to get a masters degree at Harvard from Birkhoff, so there's an interesting link, Poincare to Birkhoff to Lorenz.

So he studied and got this Master's Degree, and he's all ready to go on to become a mathematician, but it's 1938 and you know what's coming, WWII. It broke out, and Lorenz went into the Army Air Corps, and finds himself working as a weather forecaster. After the war, he continued to be interested in weather forecasting. He wanted to help develop the mathematical theory behind it, which was at that point, in a pretty rudimentary state, because weather forecasting had been something like black magic. It still may even to some extent, have some of that. It's very tough to predict the weather!

The best people in meteorology knew that, so didn't want to go into forecasting, and there were other problems to work on. Yet forecasting was still like reading tea leaves, so people felt this was not something that a really talented person should go into. But Lorenz didn't agree.

There were three approaches at the time, to this problem of weather forecasting. Let's just outline what they were, since they'll be relevant to our story. The first is what one can think of as dynamical forecasting, in the spirit of Isaac Newton. That is, we're going to write down the correct differential equations for the atmosphere, pressure, temperature, humidity, everything. This is enormously complicated, and much harder than the three-body problem.

Then, y solving those differential equations on computers, we'll try to inch the weather forward on instant at a time, with the problem being very difficult because not only are there all these variables, but you have to keep track of the weather at every point on the map, in three dimensions. So it's really tough. So anyway, that's one approach, though it wasn't tremendously successful, especially with the early computers of those days.

Another approach that was common was one we may think of as pattern-matching, which is one looks for weather that reminds you of that you re seeing today. So you see what happened the last time you saw weather like this. OK, so that's a kind of common sense way to do it, basing forecasts on previous experience.

A third way, which was a kind of newcomer and was challenging these other two, was statistical weather forecasting. This might sound ridiculous, just a mathematical version of tea leaves, but what you would do to calculate the temperature in New York at 4:00, is write down some formula, like maybe the temperature in New York is one third of that in Cleveland, four hours earlier, plus one eighth the temperature in St. Louis, ten hours earlier, plus a certain fudge factor for the humidity in some other place. You would have all these numbers that you could adjust in front of all the possible variables that you thought mattered. You adjust them until they give the best fit, and then that's your model. It doesn't have any physics in it, it's just totally a fitting curve. So this is statistical forecasting, this third way of doing things.

Now what Lorenz was primarily interested in, the problem he set himself, was if these statistical methods are any good? Could he test them? Yet not just against real weather, so he thought up a clever trick where he would create artificial weather on a computer, and use it to test these statistical forecasting methods. So he made a surrogate for the weather, that will be simpler for the weather, and maybe he can use it.

The advantage is one can run hundreds of days of weather very fast in the computer. You can see if the statistical methods, treating this as if it were real weather, will predict what really happens in the computer, and so on. So this was a really cool and novel strategy, because Lorenz was using the computer in a new way. It was now an arena for thought experiments. See? It's not just a calculating machine, calculating the solutions to differential equations or things like that, but it's a place where you can play games and do thought experiments. So this was already a bit of interesting creativity on his part.

Well another thing that was unusual was that Lorenz was doing so many of these computer experiments that he felt it was worthwhile to have a computer in his office. That was totally unusual, since it was the early 1960s, and people would go to computer centers with some giant machine in a big room. Yet he had a kind of personal computer of the era, called the Royal McBee, and it was very unusual. Personal computers as we know them were 25 years away.

So what happened when Lorenz did these computer experiments? The trouble was that it's hard to make a good model of the weather in a computer. His first version of artificial weather was too simple. It would always settle down to some kind of equilibrium state where it wasn't changing, which isn't like the real weather, or it would change, but in too simple a way, always repeating after a while. So it would be periodic, just repeating in perfect cycles. That's not the way the weather works either.

So he realized that what he needed was to concoct some kind of a system that he could solve in his computer, which had to be a deterministic solution of differential equations. Yet it shouldn't be periodic, it should never repeat. Otherwise the artificial weather would just be too simple and wouldn't provide a good test of these forecasting methods he was trying to study. So he had to struggle for a long time to come up with a deterministic system that would do this, not repeat itself, and eventually he did.

What he had forgotten to put into his model, was the geographical effect of the heating. Now everyone knows that it's hotter near the equator than up near the poles, so one would want to have some kind of warming that depended on latitude. Certainly he knew that, but what he didn't put in at first, and what turned out to be the missing ingredient, was that there's also a heating effect that goes east to west. That is, the heating over the oceans is different from the heating over the continents, which also makes a difference too.

So when he included that east to west, or longitudinal aspect, as well as the latitude, then he found he could get a system that was deterministic and would not repeat itself. That was what he was looking for.

Now where is the chaos in this? While studying his artificial weather, Lorenz happened to cross something that we now call the "butterfly effect." This refers to the extreme sensitivity of a chaotic system, to tiny changes in initial conditions. We call it the butterfly effect, because there's a sort of image you have in mind, of a butterfly, hypothetical of course, some imaginary butterfly, flapping its wings in Brazil, creating little air currents whose long term effect is to ultimately trigger a tornado in Texas. That's the idea. So Lorenz found something like this in his computer simulations, and he certainly wasn't looking for it.

Here's how the discovery came, and it's certainly a great story. He was re-running a simulation that he had done. People used to love coming to his office to watch his artificial weather blowing by. Of course there were no computer graphics and you were just seeing numbers and charts going up and down, but still!

So he was re-running one of his simulations and he tried to project farther into the future than he had. So what he did was run the simulation up to a certain point, and then stopped it. He recorded the numbers that the computer was printing out, wrote them down, and then used those numbers as a new starting point to then integrate even farther forward into the future.

Well of course, these were slow computers, so he knew it would take awhile. He put in these numbers that had occurred mid-way though the first simulation, typed them in, and then went down the hall to go get a cup of coffee. We know this, he wrote and told us this story. It's in his book he has written. So he goes down the hall for coffee while two months of artificial weather have now gone by, when he gets back.

Great! So he comes back and looks at the printout of the new extension of the earlier weather, and he finds that it doesn't look anything like the weather he was seeing the first time. This seems really weird, because he started it mid-way through an old simulation, which should have given the same weather up to the end, and then carry on from there. Yet it didn't! What the heck is gong on?

Well Lorenz wasn't really as shocked as we might be, since this kind of thing used to happen to his computer a lot. First of all, he suspected that it could be a weak vacuum tube. Tubes were blowing out, and it was tough in those days. It could be that, so looked all through there, yet he didn't see any problems with any tubes. It could be bugs, literally. Bugs used to get into computers and cause trouble. It didn't seem to be that, since the computer was clean. So he really was puzzled and started scratching his head over it, and thought that he can't see what the problem is with the machine.

So maybe if he can't localize the problem, he could figure out where in the computer printout the deviation first occurred. This might help because he could then give the information to the repairman, which he knew from previous experience, might help to speed up the fixing of the machine.

So he starts looking at the printouts, the old one and the new one, and he sees that there is no sudden mistake. That in fact, what they had first thought, that they disagreed, wasn't really true after all. They agreed perfectly at first, but then as he's sort of tracing through the numbers, he sees that one of them differs by one, in the last decimal place. Then he keeps going down the column and now it differs by two decimal places, then three, then four in the next decimal place. The error is growing as he goes father into the future, with the discrepancy between the old run and the new run, doubling in size about every four days.

So that was a clue. Can you guess what the problem was? Well the clue revealed the culprit. The initial conditions he used when he started the second run, mid-way through the first? They had not actually been the same in the two simulations. Why not? He tried, and he wrote down the numbers, but why not? The reason is that, what it was printing out on his computer was three digits, like .628 and numbers like that. He needed to do that to save space on the printout. So he would printout just three digit numbers. Yet in fact his computer, in its internal arithmetic, was computing with six digits, like .628172 for example. Those were just getting lopped off and truncated.

What had happened then, is that by ignoring those truncated terms that he couldn't see in the printout, Lorenz was creating round off errors in the fourth decimal place. These were growing exponentially fast, and ultimately overwhelming and changing the weather itself. That little fourth decimal place is like the butterfly, and that changed the weather.

So lets now see a modern computer simulation of this effect, and keep in mind that it's much easier for us to do what we're about to do, than for Lorenz, who was just printing out numbers on a slow computer and so on. We're going to just see a real computer graphic of this, which is very fast.

To be honest, the simulation we'll see is not exactly Lorenz' original weather model, but his later model, the famous Lorenz equations. These are much simpler than his original weather model, which only uses three variables, where this uses twelve, but it's basically the same idea and shows the same butterfly effect. So Steven feels like he's not really being scandalous in this respect by using this one!

So what we'll see in this simulation, just to get ourselves oriented, is the standard kind of graph called a time series. We'll see a variable, which in this case represents the spin of a certain water wheel that we'll discuss in the next lecture. So there's some variable of spin as a function of time, and we'll see it bobbling up and down. OK? So watch the graph.

We're going to start the solution from some initial condition, and there it goes. So we see a variable that at first is oscillating, and appears to to then be growing. They get bigger and bigger, like a large amplitude sine wave, and then whoa! Something different happens and now it's starting to look like its not repeating. It's aperiodic, which is the osrt of thing that Lorenz wanted, remember, one that wouldn't exactly repeat.

So we're seeing some kind of aperiodic signal, which after all is what he was after. He wanted to make non-periodic behavior from a deterministic system. There's an example of such a thing. Next we'll what happens when we start mid-way through the curve we just saw. We start mid-way through the simulation, and wheres we had been keeping six digits of accuracy, we'll now only keep three.

So when we hit to restart button, we'll see a new curve that at first will track the previous curve, right on top of it, just like Lorens' numbers agreed at first. But then we'll see it diverge. So the new curve starts out right on top of the old one, but then diverges and has a totally different future. The difference was that the new curve was the truncated version in which we kept only three digits (.506) from a number that was (.506127) originally.

So this is what the butterfly effect looks like. Good agreement up to a point, and then radical divergence after that. Allright, so back to our story then. What are the broader implications of Lorenz' work? We have this butterfly effect, but really, so what?

Well the butterfly effect is the signature of chaos. This is the thing that defines chaos practically. There is a deterministic system which is unpredictable in the long term, because any tiny error gets amplified exponentially fast, and in this case made the artificial weather unpredictable.

The implication of this is that the real weather, which is surely more complicated than what we're seeing in the computer, is also very likely to be impossible to predict into the long term. Now we have to talk about how long term actually is long term? That will be discussed a bit in the next lecture. But for now that's a quantitative detail.

The real point is that even if we had a perfect model of the atmosphere, and knew all there was to know about the laws governing it, the inevitable errors in our measurements of the current state of the atmosphere, would very quickly grow, snowballing until they make any forecast look silly.

There's also an interesting lesson here about the discovery process, which holds broader lessons for science than just chaos. Lorenz was serendipitous in discovering the butterfly effect, and that's not the same as being lucky. He didn't discover it by accident, but by serendipity. The distinction is that Lorenz was not just blundering around, but was deliberately looking for something, a non-periodic system that was deterministic to test a forecasting method. So what that did was make his mind alert. He's looking for something, he's keenly vigilant, he's ready to see trouble or anything unexpected, and he did. It just turned out to to be something he wasn't looking for, and was much more interesting that what he was trying to do, it was chaos. So it was because of his frame of mind. You know that famous line from Pasteur about chance favoring only the prepared mind. This is the way one makes scientific discoveries, by serendipity.

Well, also, Lorenz' research strategy changed everything about the way we do modern chaos research. He adopted Poincare's pictorial approach, as we'll see in the next lecture, but he strengthened it enormously by using a modern computer to simulate the system and graph the results. Instead of advancing the simulated weather frame by frame with formulas the way Newton would have tried to do, by solving a differential equation, or by picturing state space in his head the way Poincare had to, not having computers, Lorenz had the computer grind forward, one instant at a time, pushing those differential equations and their solutions forward.

So with the computer as a weapon for solving or attacking problems, it could go much father than Poincare had. Yet really, that's not the big innovation. It's the way he used to computer, as not just a number cruncher, but a sort of mind-amplifier, a telescope for the mind that led him to imagine the inconceivable. So finally scientists could see the consequences of the laws of motion, even though they still couldn't write formulas for those solutions. Computers were now giving intuition.

So in retrospect, Steven thinks this is the answer to this question, of why did chaos theory have to wait until the 1960s to really get going? Without the computer to form millions of calculations in the blink of an eye, scientists couldn't begin to fathom what their equations were trying to tell them.

So in the next lecture we'll take a step back, and think about the butterfly effect from a more philosophical perspective, and ask questions like what this tells us about the course of our own lives? What about destiny, fate, and making sense about a chaotic world, given that we can't really predict it? Well stay tuned.