Management Lessons of the Moon Program (Andrew Chaikin)

>>MR. ROGERS: Let me introduce our -- our speaker at -- in the afternoon, Andrew Chaikin,
space author and historian, and I must add personal friend.
I've really enjoyed getting to know Andy over the last few years working very closely with him.
Andy is an award winning scientist journalist
and space historian, he has authored books and articles about space exploration
and astronomy for more than 25 years.
Writer, director, and explorer, James Cameron of the Titanic and Aliens of the Deep
called him our best historian of the space age.
Chaikin is best known as the author of Man on the Moon, a Voyage of the Apollo Astronauts,
many of you have a copy of that, widely regarded as a definitive account of the moon’s missions.
First published in 1994, this acclaimed work was the main basis for Tom Hank’s 12 part HBO
series, From the Earth to the Moon, which won the Emmy for best miniseries in 1998.
Chaikin spent eight years writing and researching the Man on the Moon
including over 150 hours of personal interviews with 23 of the 24 lunar astronauts.
A Passion for Mars was published in September 2008 by Abrams,
Apollo 11 astronaut and author Michael Collins called it,
“A masterpiece of a book” and rocket boys author Homer Hickman said,
“I am completely and utterly in love with this book.”
Chaikin collaborated with moon walker turned artist Allan Bean to write
Apollo: An Eyewitness Account published in 1998 by the Greenwich workshop press.
He is currently working on a book about management lessons
from NASA’s first 50 years and he's here to share some of that insight with us,
particularly around the Apollo problem.
Please welcome author, journalist, space historian, Mr. Andrew Chaikin.
( APPLAUSE)
>>MR. CHAIKIN: Thank you Ed, thanks for having me.
Quite honestly it's an honor to be part of this I'm very grateful to be able to do the work that I do,
having grown up with space exploration as my passion in life.
And you know those of you who know my work, know that I'm pretty heavily into Apollo,
and you know I'm looking out at the auditorium,
it's filling up nicely now, but before I was thinking you know I wonder if that drives some people
away, you know my God Chaikin's going to talk about Apollo again, but when you --
When you get into the story of Apollo and I really kind of turned myself into a story teller of space,
you rapidly discover that the story of Apollo is one of the most compelling human
adventures that's ever been carried out on so many different levels, you know,
I of course when I wrote a Man on the Moon was writing about it not only from a technological
and scientific level but from a human level through the eyes of the astronauts
to tell that story ,but since I hooked up with Ed,
and have been working with him on the book that he mentioned,
I've really discovered that there is a whole story to Apollo that I didn't even really know,
which is every bit as compelling in different ways.
And I want to try and share with you some of what I learned in the process of delving into that aspect of Apollo.
You know Apollo has been called the greatest technological achievement of the century,
even the millennium, and when you look at this picture you really understand why,
this is sort of a scale model that I constructed that shows the Earth all the way down here
and then way up here is the moon.
Sometimes I go into schools and I ask young people, I have one of them hold up a --
a model of the Earth and then I have a model of the moon that's to scale, and I --
I walk away from the other student I say stop me when you think I'm as far away as the moon
really is and they usually stop me a few feet away and then you know no, not yet.
I keep walking I keep walking and I end up on the other side of the room,
and you know most people don't even realize how far away the moon is.
It's 30 Earth diameters away, it's 240,000 miles, so
imagine even for veteran NASA engineers in 1961 when president Kennedy,
who was the CEO of the country,
I mean talk about a job assignment he gets up before Congress
and he says not only are we going to go to the moon but we're going to do it within the decade,
and one out of many reactions, this person said to me when I interviewed him,
referring to Kennedy now, I thought he'd lost his mind.
I didn't even think about the technical requirements,
I just knew it was a job of fantastic magnitude.
Now in case you're wondering who that was and thinking well maybe this was somebody
who just wasn't familiar with the groundwork that had already been done
and the thinking about how to go to the moon, not so much because it was Chris Kraft.
And so this is to show you that even the people
who were on the front lines of human space flight were completely taken aback by Kennedy's
challenge and it's understanding when you realize that at the time Kennedy
made that speech on May 25th, 1961, we as a country had exactly 15 minutes
of human space flight under our belt.
It was basically an up and down sub-orbital hop by Allen Shepard, but we did it, right?
We made what really you know understandably is called an impossible dream come true.
And over the years people have focused on the courage of the astronauts,
they've focused on the incredible technical advance that were created for Apollo,
but I want to show you something that was written right before Apollo 11 landed on the moon,
in of all places Fortune magazine,
but the guy that wrote this article that I'm about to quote from was a science writer named Tom
Alexander, who had been following the development Apollo program all the way along.
And he actually wrote in Fortune magazine in the July 1969 issue
the really significant fall outs from the strains, traumas,
and endless experimentation of project Apollo has been of a sociological rather
than of a technological nature.
Techniques for directing the massed endeavors of scores of thousands of minds,
this is potentially the most powerful tool in man's history, that is quite a statement.
In fact, a lot of people felt that way, people within NASA, people like Jim Webb,
the administrator of NASA, and a lot of people thought that this was such a powerful tool
that you could now use it to solve the world's problems, right?
How many of you have heard the phrase, if they can put a man on the moon why can't they --
fill in the blank, right?
Everything from curing cancer to solving the problems of inter cities,
turns out it's not quite so easy to take a management solution
for what is basically a gigantic engineering project
and apply it to social problems and other kinds of non-engineering things.
But this is really a profound statement. So what I'd like to do is try and break that down.
And as I have done my research and thought about this
and believe me this is not to say that I've covered the, you know, the be all and end all list, right,
some of you either here or watching on the web may come up with other ideas
and I'm sure I'll hear from you, but this is my take on -- on the --
the principles behind Apollo's management triumph, okay?
Number one, a clear and compelling goal that comes from the top.
Number two, sufficient resources to accomplish that goal.
Number three, a systems approach to managing complexity and I'll explain what I'm talking about when I get to that.
Number four, the optimum solution could win, even against powerful opponents, by the way.
Number five, you reduce risk by designing for simplicity and redundancy.
Number six, test, test, test, but test under the conditions in which you fly.
You may have heard the phrase test as you fly, fly as you test,
it sounds like you're doing everything at the same time, but that's not what it means,
it means test the way you fly and fly the way that you test.
Number seven is what-if thinking, one of the most important bullet points on this whole list.
Eight, accountability at all levels of the program. And number nine may surprise some of you,
but maybe not, luck. And we'll get to that and I'll explain why I say that.
Okay, let's go through these.
Kennedy got up before Congress and gave arguably one of the most clear
and compelling goals that any leader has ever given.
In fact Dave Scott, who was the commander of Apollo 15
when I interviewed him in 1987, said to me probably one of the clearest
definitions of an objective or a destiny that mankind has ever experienced has been man,
moon, 1970, how could it be any clearer than that?
Now we all know that's part of the story,
we all know that Kennedy got up there and said,
very clearly, something that took people's breath away and in the case of Chris Kraft,
caused them to momentarily
doubt his sanity, but what's important for this conversation is to understand
what that does to the organization
because at any point when you come up against a difficulty,
the response to that difficulty is going to be modulated, so to speak,
by the clarity of the goal and how compelling the goal is.
And there's a wonderful quote that Charles Murray and Catherine Cox
put in their wonderful book; Apollo: Race to the Moon from
the Apollo program manager Joe Shea, "Down at Houston we would argue like hell
but after wards we would be out in the parking lot and look up at the moon and say,
'you really want to go there'?"
And of course everyone knew what the answer was.
Number two, sufficient resources to accomplish it, well that fellow there Jim Webb,
who was NASA's administrator at the time of Apollo,
at least up through the start of the lunar missions
is much of the reasons why there were sufficient resources.
Webb was, before he came to NASA, had been director of the bureau of the budget,
Kennedy asked him to takeover NASA,
he came in already knowing how to work within Washington,
how to work the complexities of negotiating with Congress,
where the skeletons were buried when he needed them.
He would try give and take and when that wouldn't work, he would use Kennedy's objective,
in the words of one writer, as a hammer.
After Kennedy was assassinated, he appealed to the sentiment
that Apollo was a fitting memorial to the slain president.
He even did something that I think must make every NASA administrator
since then sigh with wistful longing,
when he gave NASA's estimate for the cost of Apollo to the white house,
and that was the first step in the process that started the budgetary ball rolling, so to speak.
He took the figure that his people gave him,
which was 12 billion and he used what he called his administrator's discount and doubled it
and gave that figure to the white house.
Guess what? It turned out to be right on the money.
That's exactly what Apollo ended up costing.
Now you can always talk about whether that's a coincidence or not or part of human nature
that you work to spend all the money you have, but the point being that if you talk to people today,
you talk to people who have worked on Constellation
or any other program who look back at Apollo
and they say boy the one thing about Apollo was they were never hurting for money,
that was the free variable, if I'm using phrase correctly. Mike is nodding, I think I'm all right.
So you know and that's borne out if you look at the curve of the NASA budget
as a function of the entire federal budget, okay, throughout NASA's history,
in 1965 at the peak of Apollo activity when they were building up the infrastructure at the cape,
all those launch pads the VAB, they were doing all the development and testing on the Saturn five,
they were developing the space craft, all of that enormous effort NASA's budget was 5.3 percent of the federal budget.
What is it today? Half a percent.
So I think I've made the point that Apollo had sufficient resources.
Okay number three, a systems approach to managing complexity.
Now this is really where things get pretty down to brass tax,
this is where it really the rubber really hits the road.
The people in this picture are the main actors in our little story
and on the left you see the head of the office of manned space flight at NASA headquarters,
George Mueller, who was brought in in 1963,
next to him is General Sam Phillips from the Air Force, who was brought in by Mueller.
Phillips had been the manager of the minute man ICBM program for the Air Force,
and next to them you have the three senate directors for the centers
who were doing the main hardware and mission development work for Apollo,
Kurt Debus, director of KSC, Bob Gilruth, director of--
at that time the manned space craft center, MSC,
and Wernher von Braun, director of Marshal.
Now there's a little phenomenon that you may have heard of called inter center rivalry, well,
it's very instructive to go back 40, 50 years,
we're really going back 50 years, here,
and find out that inter center rivalry was alive and well at NASA, even back then.
Not a big surprise if you think about what NASA was,
it was a collection of people from different organizations that had different cultures, right?
Von Braun had led the development of the V 2 rocket for the Nazis at Pena Mundo,
and Kurt Debus was one of his people and that was a very tightly knit team,
they were brought over after the war.
Von Braun his people became the nucleus of the Marshall Space Flight Center,
Debus had gone onto run KSC, but they had developed a way of working together,
they trusted each other, they trusted each other's competence
and the competence of outsiders, maybe not so much.
Meanwhile, Bob Gilruth came out of the NACA,
the precursor organization of NASA,
was one of the giants of aeronautics,
had led the development of rocket propelled models for supersonic testing,
the NACA approached engineering very methodical, again, very collaborative,
in-house, these guys were all used to working in-house, R and D in-house.
They were not number one they were not managers, okay, they were engineers,
number two, they were not used to working with outside contractors to do their development work.
Well, that was sort of okay in the early days,
you know, mercury was designed in-house at Langley
and then given to McDonnell Douglas to build,
but you know even then there were a sign of problems.
There was a committee system so you had committees to worry
about every part the spacecraft,
but it got so complicated they needed a committee to manage the committees.
So even before Apollo got going, even before George Mueller was brought in,
they were already budget overruns,
schedule overruns and things like that and basically controlled chaos, in a management sense.
Mueller came in and recognized that here was a group of people
who needed to learn how to manage complex projects.
And so he did a lot, himself, and we'll see a little more of that in a second,
but he also brought in Sam Phillips from the Air Force to really impose the methodologies
of what we now know as systems engineering. Now let's talk a little bit of what Mueller did.
Oh, this is a quote from Sam Philips that I loved,
this is from an oral history from 1988, where he said,
"Yes, I tended to characterize NASA as a group of feudal baronies.”
These were organizations that didn't necessarily recognize the legitimacy of the other baronies
to take leadership roles and certainly they didn't want anybody above them telling them what to do.
I think some people here at Goddard may be familiar with that and Goddard's history as well.
It seems to me I've heard a quote or two about that.
Okay, von Braun, for example, just to give you an idea,
had a system of Monday notes that he would require from his people.
So people two levels below him had to send him notes
on how things were going and air problems and so forth.
The reason he did it for two levels below him was that so their managers
couldn't filter the information, this gave von Braun a direct path of communication
to the people actually working with the technical problems
and it gave him a way of responding to those issues directly.
Well, that was great within Marshall, but what Mueller did,
and I don't know that he got the idea from von Braun’s notes or not, he may have come up with it independently
but he instituted a system that was agency wide
and what he did it came to be called gem boxes because of his initials George E. Mueller.
So what he did is, here he is office of manned space flight,
he set up offices at headquarters for program control, system engineering, test, reliability,
and Q and A, and flight operations,
and then what he did was he established corollary offices,
contrary parts at each of the field centers,
so the guy at headquarters who was managing program control
could speak directly to his counterpart at MSC or KSC or Marshall, same with systems engineer,
same with test, so on and so on.
So this gave Mueller a very strong hand in monitoring the technical progress of Apollo
and addressing issues that needed addressing.
Let's take a quick look at how this fit into one aspect of Apollo,
here's the org chart for the Saturn 5 program,
the mammoth booster we use to send humans to the moon, even without the space craft.
The Saturn five was a program of startling complexity that staggers the imagination,
so even that alone was a triumph of management.
Again here you have Mueller at the top, under him is Phillips
and you have the center director him, von Braun,
this is the NASA line organization,
but separate from the NASA line organization you have this --
This gem box set up that is superimposed on it and you have daily coordination
between program control at headquarters, at Marshall, systems engineering
and so on and so on and so on.
So this really shows just one example of how Mueller imposed principals of system engineering on the agency.
Now, another aspect of that, and something that Phillip's brought in
and really considered in his own words to be the Bible of systems engineering,
is the technique called configuration management
and there was an incredible amount of resistance to configuration management. Why is that?
Well, think about who was putting up the resistance, engineers,
what do engineers like to do? They like to design and build things.
Do they like to be told that you can't make any changes? No.
They get a better -- an idea of how to make it better, they want to keep making it better.
So configuration management kind of set off alarm bells in all of these creative engineers
who wanted to just be allowed to do their work
and keep making things better and better and better and Sam Phillips trying to impose
this had a devil of a time getting people to comply.
And at one presentation George Mueller here said to the agency people,
"Look, configuration management doesn't mean you can't change it.
It means you define at each stage of the game what you think the design is going to be
within your present ability.
The difference is that after you describe it,
you let everybody know what it is when you change it."
Okay? And this was Miller's comment circa 1964,
and it still took another two plus years for that to really grab hold
and unfortunately the thing that made it grab hold was the fire that killed the Apollo one crew.
And if you look at the causes of the Apollo One fire
you find that behind the immediate causes, pure oxygen at high pressure, inadequate escape options,
flammable materials in the cabin, was a systemic problem of lack of configuration control,
the design of the space craft was being changed on a regular basis, in fact,
it was so bad that the simulator guys couldn't even keep up with it
and you've all heard the story about Gus Grissom hanging a lemon on the simulator because of that.
So it took the fire to really make people come around completely
and listen to Phillips and allow him to impose configuration management on NASA,
but it ended up being crucial also for controlling costs and staying on schedule.
And again, it doesn't mean you can't make changes.
What it means is, they had a review board at each stage of the game
and they had what we now experience as a regular part of
any project with preliminary design review, CDR,
all the way up to flight readiness review, that was something that Phillips started, okay? Okay.
The optimum solution can win even against powerful opponents.
What you see there is a drawing of the Apollo spacecraft
as it was initially envisioned at the time of Kennedy's speech.
I want you to note the height figure here of 65 feet.
I'm not talking about an upper stage that's discarded after you head to the moon,
this is the part that lands on the moon.
They were going to land a 65 foot high vehicle backwards on the moon.
I mean I get goose bumps just even thinking about it.
Well, there were already people, even before Kennedy launched Apollo
who were thinking about how to do it differently,
and many of them were at Langley and one of them was an engineer named John Houbolt,
who became the most passionate and vocal proponent of an alternative approach,
which was called lunar orbit rendezvous.
Now at the time of Kennedy's speech most people thought
that what you do is you would launch the big Apollo spacecraft on one rocket
and on a second rocket you would refuel it once it got into Earth orbit
and then send it off to the moon.
Houbolt said that's kind of nuts because you're paying for that all the way back at the other end of the rocket equation, right?
Every pound that you put on the surface multiplies back and back and back and back,
until you need this enormous rocket,
they called it Nova and they came to the conclusion pretty quickly
that they couldn't build Nova so they were going to do it with these two launches and refuel in Earth orbit.
Well, we didn't know how to do that, we didn't know how to build rendezvous.
Nobody knew how long it would be or how easy,
most people thought it would be incredibly hard.
Houbolt said, let's do a rendezvous in lunar orbit and instead of having a big massive vehicle,
let's have a vehicle that's much, much smaller and have that,
the only piece that goes down to the moon
and leave as much mass as you can in lunar orbit.
And then at the end of the landing phase of the mission, this part,
the upper part of the lander blasts rendezvous with the crew mate
in lunar orbit and then they come home.
Well he approached a lot of people, he talked to Bob Siemens, went to people all over the place.
He got a hearing at NASA headquarters in the beginning of 1961
and I want to tell you there was no problem with silence at these meetings.
One of the guys at the meeting was Max Faget, who was a brilliant, but quirk key engineer,
Louisiana born, had been the main guy behind designing the mercury capsule
and was the main guy designing the Apollo command module.
And Houbolt got up there and said,
“You know we could save two to two and a half times the mass
by doing it this way and as soon as he was done speaking Faget leapt up and said,
“His figures lie, he doesn't know what he's talking about!”
Not exactly a subtle feedback approach, here.
So you can see what Houbolt was up against,
but he managed to convince MSC that in fact his analysis was valid
and then it came down to Wernher von Braun.
So first he got Faget tamed and then von Braun so there was a meeting
with all the senate director and then von Braun surprised his own people
after listening to these presentations, saying we're going to go behind lunar orbit rendezvous,
even though von Braun’s own vision was you would do Earth-orbit rendezvous
and that would give you the knowledge base to go forward and do things like a space station
and building Mars ship in earth orbit and all these grand things he wanted to do.
He understood that the orders from the top,
again that clear compelling goal,
was to get to the moon by the end of the decade
and the only way to do that was to simplify things with LOR.
So the optimum solution could win, it could get messy at times, right,
people yelling at each other, but one of the great strengths at NASA was people could have it out
and change their minds if they were presented with data that could convince them.
Here's one more example, the lunar module was the last piece of Apollo to be developed,
that part that goes all the way down to the moon,
and because of that, it paid the greatest weight penalties
and the struggle to keep the lamb’s weight under control was ever present,
it was absolutely grueling for the guys at Grumman, who were building the LEM.
And one of the problems was that the engine here would be used to land on the moon
had a supply of helium that was used to pressurize the propellant tanks and that helium- -
Those helium tanks were actually kind of --
kind of massive, there were two of them, and someone at Grumman,
a thermal engineer came up with an idea for something called super critical helium.
What the heck is hypocritical helium, it turns out if you take helium gas
and chill it down to minus 450 degrees Fahrenheit, there's a quantum dynamical effect,
I kid you not, where it turns into something that’s between a gas and a liquid,
which can be compressed to very high densities.
Well Owen Maynard, who was at MSC and was leading the development of the LEM,
heard about this and he thought it was a pretty darn good idea.
So he began to include it in the planning and you can see the super critical helium tank
saved them anywhere from 250 to 300 pounds
by reducing the number of tanks from two down to one and making it—It was a vacuum jacketed
tank and they could compress it to eight times the density of ambient helium.
Well, there was a program review of the lunar module in I think 1965
and Owen Maynard had been pushing this idea and guess who,
Max Faget took exception to this and in a public, not public,
but within NASA all the program managers were standing around.
Sam Phillips was there, Sam Phillips looks over and sees Faget shouting at Owen Maynard,
“What is this garbage, you're asking me to stake my life on this crazy thing?
I'm not going to do this.”
Things could get messy, but essentially Faget saw the wisdom of it and that's how they went to the moon.
It was the optimum solution for the conditions they had.
Okay designing for simplicity of redundancy,
many of you know this story so I won't spend an inordinate amount of time on it,
but you know basically Faget believed and many people
believed that you reduce risk at the design phase, okay,
you design in safety and one of the ways do you that is simplicity around the other way is redundancy.
Now, we could have a very interesting conversation
about whether redundancy really simplifies things or actually adds complexity,
increases the number of failure modes, etc.
I'm not equipped to have that conversation nor do I have the time so thankfully I'll sidestep that,
but I will point out that there was quite a bit of redundancy in Apollo
and we're looking here at the mother ship, the command module,
and the service module and in the service module alone you had redundancy in things like attitude
control thrusters and the circuitry that controlled them and so forth and so on.
Then the fuel cells, which provided electrical power
and also drinking water, they were redundant and the tankage supplying them was redundant.
So you know many, many times
where you can go through the history of Apollo and redundancy saved the day.
But what about where you couldn't have redundancy,
well what about things like this big engine, this service propulsion system
that got the astronauts into lunar or built and more importantly
out of lunar orbit when it was time to come home there Earth.
Well, there simplicity was the order of the day so they designed an engine that needed no ignition system, why?
Because it used hypergolic propellants that ignited on contact.
It didn’t need fuel pumps because it was pressure fed.
There were redundant ball valves that would make sure that the propellants got together
in the combustion chamber and also an ablative nozzle,
without regenerative cooling because again simplicity.
That engine worked beautifully, so did the other engines that used the same design approach,
the descent engine and ascent engine in the lunar module.
Okay, but part of the reason they worked so well was
because so much effort and expense was directed at testing everything
that flew and testing it under flight conditions
and in fact George Low, who was the Apollo spacecraft program manager
and one of the most brilliant engineers in NASA history said,
“The single most important factor leading to the high degree of reliability of the Apollo spacecraft
was the tremendous depth and breadth of the test activity.”
Okay, and let me take a moment here to point out that test was really one of the reasons
why we were successful and the Russians weren't.
If somebody ever asked you how come the Russians never beat us to the moon,
there are a lot of reasons for that, and I'll give you some more of them in a minute,
but one of the main ones is they didn't have the resources to perform the necessary ground tests.
In fact they didn't even have the engineering philosophy that said,
test as you fly, fly as you test.
They were more treating it like the old artillery programs
where you just fire them off until you get it right, right?
Well it turns out when you're building a lunar vehicle that costs millions of rubles,
that approach doesn't work too well politically
and they lost every one of their giant moon rockets
because they hadn't tested the thing properly.
Okay, this is one of my favorites because I think it's --
It's one of the things that translates most readily to any form of human endeavor,
I use it in my own life constantly, and that's what if thinking.
It's the process of sitting down, even in an environment
as benign as a conference room, and saying, okay,
we're 200,000 miles out in space and something goes horribly wrong, what do we do.
Well, that happened on Apollo 13, didn't it t and they ended up having to use the lunar module,
which was attached as a life boat to takeover for the crippled command service module.
Turns out that a lot of the stuff they did on Apollo 13
had been thought about to some degree long before the flight.
For example, the rocket firing that was used to get the astronauts back
on their so-called free return trajectory, to get them back home to Earth
was performed by the lunar modules descent engine,
the engine that would normally be used to land on the moon.
Well that had been tried on Apollo 9, the earth-orbit test flight of the
combined command service module and lunar module.
They had done this as a docked burn in Earth orbit.
Okay, here's another one, when they were on their way back from going around the moon
and coming back towards Earth, they had to do a correction to their flight path,
they didn't have their computer up and running because they had to save power.
So how the heck do you make sure the spacecraft is in the proper alignment
to do the burn that you've got to do to get you back in the re-entry corridor?
That's a pretty precise thing and you better get it right.
Well, there was a technique in which the astronauts
would look through their optical alignment site,
which had a grid on it and they point the spacecraft at the Earth
and they would maneuver the spacecraft,
until the horns, the points of the crescent Earth were sitting on this horizontal line
and centered according to the vertical line.
That technique, which they did on Apollo 13,
had been thought up for Apollo 8,
the first flight around the moon in 1968 as a contingency procedure.
Now, it is true that when Houston told them they were going to have to do this,
Jim Lovell very well thought to himself, boy, I never thought we'd have to use that one thought
it -- I hope it works. Yes, that's true, but the fact is it did work and again it had been thought up before.
And finally the much celebrated work by the crew systems guys
and the guys in the mission evaluation room to develop,
from materials that they had on board a way of adapting the lithium hydroxide canisters
from the command module to the environmental control system in the lunar module
so that they could continue to remove CO2 from the atmosphere when it rose to dangerous levels.
Well, I won't say that it had all been thought up before,
but it had partly been thought of as a result of an Apollo 8 situation
in which they failed the cabin fans.
So I would say in summary of Apollo 13 that yes,
it was an incredible real-time effort,
but I would say that it is actually a combination of what-if thinking long before the fact,
combined with real-time ingenuity that saved those -- those
astronauts and allowed them to come home.
Okay, one more example of what-if thinking and this is actually stuff that was not used.
The later Apollo missions went farther away from the lunar module
and on Apollo 14 when they walked a couple of miles,
they came up with a safety plan in case one of the astronauts backpacks failed,
there would be an emergency supply of oxygen in the oxygen purge system pack
which was on top of the main backpack.
So oxygen was not the issue, the issue was cooling water.
Normally the backpack circulates cooling water through tubes in the long under wear
that the astronauts are wearing and keeps them from overheating,
which would happen extremely fast if you didn't have that cooling system.
So they came up with a buddy hose to share cooling water.
If one of the backpacks failed, they would hook it up the buddy hose,
which was in a bag attached to their tool carrier, they would hook it up,
go high tailing it back to the LEM.
Now the missions, the last three missions; Apollo 15, 16, and 17
had a rover and so you were going even farther away,
three and a half miles away, four miles away and in that case you had to think
about what would happen if the rover failed
and you had to walk back and what they did there was they came up with something
called the walk back limit and that meant that you design the moon walks
so you could go to the farthest point that you're going to first.
And so this is an example from Apollo 16 where they drove three plus miles
up the side of a lunar mountain, a few hundred feet up the side of a lunar mountain
to station four here and they did that first.
So if they had a rover failure they would have more than enough oxygen to walk the three
and a half miles back to the lunar module,
even in 1.6 gravity that's quite a chore in pressurized space suits
and then as you get progressively closer to the lunar module on the successive geology stops,
it's okay that you have less oxygen than you did before.
Now, maybe you're thinking what happens if you have a backpack failure and a rover failure,
that's a bad day, there was nothing they could do, somebody dies and so they just –
There is only so much you can plan for within the resources you have.
Accountability, boy is this one is key.
Chris Scolese said to me once,
“Accountability is the hallmark of what we do at NASA.”
And there are so many stories where accountability comes through during Apollo.
One of my favorites was told to me by Ken Mattingly,
who was the command module pilot on Apollo 16
and he did something that in this picture we see his follow on command module pilot Apollo 17,
Ron Evans, he's in the inter stage between the second and third stages of Saturn Five
and he’s talking to one the technicians for the third stage for McDonnell Douglas.
Mattingly did that not long before his Apollo 16 launch
and at first the guy didn't know who he was and told him to get the hell out.
Mattingly explained who he was and the guy said, “I'm sorry.”
And then thought about it for a minute, you know I can't imagine what it's going to be like for you
to do this to ride this thing and go to the moon,
but I can promise you that it won't fail because of anything that I do.
And Mattingly understood at that moment that the reason we got to the moon,
and we did it several times, all those times successfully was
because 400,000 people at NASA and at the contractors around the country
were saying to themselves it won't fail because of me. That's accountability.
Another story that I came across recently, I didn't know this one,
happened when one of the guys at Marshall screwed up,
I think he connected two wires wrong or something like that
and I think it caused a test failure he went right to his manager and said I screwed up, my fault.
Von Braun bought him a bottle of booze and drank it with him.
You don't want -- the take away there is you want people to be encouraged to come forward
when they crew screw up and you can hear people talk about the openness of the organization in Apollo.
Now I'm not saying they were perfect,
I'm sure there are plenty of other times when people could have been forth coming and weren't,
but I don't think there were too many.
Okay, finally, luck. Now why do I have luck on here? Let me give you one example.
George Mueller, who was the chief of the office of manned space flight at NASA headquarters,
decided- - he thought about the test program that Marshall had planned for the Saturn Five
and the way the Germans were used to working very methodically
is that you would have one stage on each test flight added a live stage.
So on the first flight you would have a live first stage and dummy second and third stage
with just water in them for ballast.
And then if that worked, you would you do on the next flight,
you would add a live second stage, and a live third stage and so on and so on.
Well the Germans did that because they thought it was a way of reducing risk, right,
this was the conservatism of the German design philosophy and test philosophy.
Mueller came from a completely different background of systems engineering
and he thought to himself, no that's wrong.
He said, “You-- all you're doing is wasting hardware and more importantly
since they took three months for each launch you're wasting time.”
So he came out and said we're going to test fly the Saturn Five complete the first time we fly it.
He called it all up testing, and von Braun said, no, you're not,
and Mueller said, yes you are, and guess what, Mueller had the power to dictate,
“Thou shalt do all the testing of the Saturn Five.”
And his logic was sound, but would we be saying that today if the first test launch of the Saturn five,
Apollo 4, had an exploded during flight or something else, the way the Russian launches did.
I don't think we would be saying that and Mueller might not have been in his job much longer, we don't know.
The fact is everything worked and Mueller himself has admitted that yes he was lucky.
Okay, here's a couple more examples from Apollo 13,
we don't think that as an example of luck going into the astronaut’s favor,
but there's a Apollo veteran engineer, Jerry Woodfill,
who worked in the mission evaluation room where a lot of the creativity,
the real-time creativity happened during Apollo.
And if you want to read this in more detail I suggest you Google 13 things
that saved Apollo 13.
If you Google that, you'll find a series of articles that Jerry wrote,
but he explains how luck actually played a pretty key role in Apollo 13’s rescue.
Here's one, okay the reason that the explosion happened on Apollo 13
was because there was an oxygen tank that had been damaged years before,
they thought they fixed the damage, it then didn't behave quite right,
so they developed a work around procedure for draining the oxygen out of the tank after a test
that involved heating it up with heaters inside the tank to boil it off.
And the heating of the tank was controlled by a thermostat.
Unfortunately, when they changed the operating voltage at the pad halfway through the Apollo
program, nobody managed to get this message to the guys that built the thermostat
and when they applied the new voltage to the thermostat it --
It completely fried itself
and stuck open and so some number of weeks before the Apollo 13 launch,
they had to use that work around procedure to boil off the oxygen in the tank.
They ended up heating the interior of the tank to about a thousand degrees
and wrecking the installation on a couple of wires attached to the fan that stirred the cryogenic oxygen.
Are you with me so far?
Okay. Now, flash forward to the flight,
there is a normal procedure that they use called stirring the tanks
and if you saw the Apollo 13 movie you saw Bill Paxton and Kevin Bacon
arguing about who stirred the tanks and screwed up and caused this mess that they were in.
The reason that they did that was because it gave them more accurate readings
of the quantity inside the oxygen tank.
Well, it turns out that there was a problem with the quantity sensor on oxygen tank number two,
and so instead of mission control asking them to stir the tanks once every 24 hours,
which was the normal timeline, they had them stir it five times in 55 hours.
And on the fifth time, there was a spark and then an explosion in that tank,
and that's what caused the accident.
If there had not been that problem with the quantity gauge
and they had done the normal timeline of stirring the tank,
the fifth stir would have taken place when Jim Lovell very well and Fred Haise
were on the surface of the moon waking up for their second day of exploration,
Jack Swigert in the command module, would have died in a matter of you know a few hours
because he would have lost electrical power and he would have had a dead ship on his hands,
and Jim Lovell and Fred Haise would have no way to get home,
even if they could get off the surface of the moon and get back there.
So everybody would have died, so that's piece of luck number one.
Here's number two, the tank that blew was tank number two,
which was on the outboard part of the little shelf that held the oxygen tanks.
There were two of them and oxygen tank number two was the one right at the outside
of the circumference of the surface module so when it exploded the force of the explosion blew
off a panel and most of that force went outward.
What if it had been oxygen tank number one instead?
That was the one that was closer into the center.
In that case Jerry Woodfill says, the explosion would have caused tank two to explode as well.
Now you're doubling the force of the explosion, that's probably a fatal event at that point.
It would certainly blow the commanded service module apart
and do God knows what damage to the command module,
but more than likely it would have killed the crew.
And then finally, if the same exact thing had happened on Jim Lovell's first lunar mission,
Apollo 8, everybody would have died
because there was no emergency life boat, no lunar module to use to provide power and oxygen and so on.
So and there are even more which, I refer you to that article,
so that's how luck gets into the picture, here. Here's the pyramid of Apollo success
that I have humbly offered, somebody said you really ought to turn this thing upside down
because without a clear goal as the foundation for the program,
you really can't do all the other things and same with the resources and so on,
but you get the idea and luck is down here in its own little place.
And again just going back for one minute to the Soviet Union program,
they did not have the clear goal from the top, in fact,
they didn't decide to get into the moon race until three years after we announced it, 1964 was when they started.
They didn't have the resources to do it right.
They didn't have the systems approach, the clear communications,
there were all these different organizations that weren’t communicating with each other
or with management, on and on and on and on,
there are so many reasons why these blocks of this pyramid, these levels of this pyramid,
worked for us and weren't done by the Soviets and they didn't have a chance.
One of my favorite stories that I heard from Max Faget,
was that years some years after the end of Apollo,
he was walking along the beach at Galveston
with Bob Gilruth and Gilruth said, you know Max someday people are going to try to go back
to the moon and they're going to find out how hard it really, there was there was a big full moon,
overhead and they were walking along the beach at Galveston.
I heard that story and as I was thinking about it for this talk,
I thought about a painting that I saw in one of the geologist who trained the astronauts to go to the moon,
Caltech geologist named Lee Silver,
he had this on his office wall
and pointed it out to me and said what do you notice about that picture?
He said, “The ladder doesn't reach the whole way.”
We have to make the ladder reach the rest of the way with our ingenuity and our passion
and our persistence, that's as much what got to us the moon as anything else.
And you know today I'm not here to give you some message that you know the guys
who did Apollo are brilliant and NASA's not like that anywhere, no I'm not here to say that at all.
There's incredibly, smart talented people at NASA and have been all the way along.
What I'm here to say is that the environment is so much more complex than it was during Apollo,
and that all of the pressures that people are under, where the goal is not necessarily clear,
where the resources are not enough, where cost and schedule
and political pressures prevent the optimum solution from winning
or from simplicity being used to reduce risk so on and so on and so on.
And where as we've seen in the years since Apollo,
even accountability at all levels of the project, if it's there, and even luck have not been enough to produce
-- to prevent failure in those situations.
So you all have a very, very tough job trying do what they did,
what you do now, in fact I think Sunday night we're all going to be thinking very intensely
about all of these principles of success as we watch curiosity hopefully land successfully on Mars.
I think I'm going to be watching like this,
but you know I wouldn't for a moment want to step into your shoes and try to do your job
under the pressures that you're under, but it's good to talk about it
and good to think about these lessons and good to try to figure out how you can apply them now.
And I'll just close with what I think is the bumper sticker from what I've been saying,
and I'm just going to put this out here and again I'm sure I'll hear from some of you if you have
a way of making it better, okay,
“In the huge complex group endeavor that is space flight,
human nature is as critical as engineering principles.
Neither can be ignored without inviting failure.” Thank you.
(APPLAUSE)
Couple questions. We can take a a couple questions.
>>MR. ROGERS: If there's one or two burning questions we'll take those and go ahead and set up --
>>MR. CHAIKIN: To be fair I didn't actually get up to the podium until 7 minutes after 1:00.
>>MR. ROGERS: I will relinquish my five minutes to the honorable gentleman from Vermont.
So go ahead we'll take maybe two questions here
and set up take another one while we're setting up.
>>MR. CHAIKIN: I'll be signing books out there if you want to ask questions. Microphone on?
Just shout I'll repeat it.
>>AUDIENCE MEMBER: Does the human space flight program now have clear goals
and sufficient resources in a way commensurate with Apollo if not do they reflect (INAUDIBLE).
>>MR. CHAIKIN: So the question is does the human space flight program have clear goals
and sufficient resources commensurate with Apollo and if not does it reflect the more complex
environment that we have now, well my answer is no to the first one and yes to the second one.
We haven't had a goal as clear as Apollo since Apollo
and I think that's caused a lot of the problems that we've seen.
We haven’t had sufficient resources for the goals that we've set ourselves.
NASA was only given about half of what it asked for, so to build the shuttle,
so the administrator's discount went the other way and that shows you how different things
were in 1972 when Nixon approved the shuttle that it had been in `61 when Kennedy launched Apollo.
I think my own personal point of view is that NASA's at a crossroads
where they are trying to shift paradigms about how you do human space flight and it --
That is always a painful process and it's painful for people within the agency
and it's painful for all of us who love NASA and want to see it succeed on the outside.
So I think it's going continue to be somewhat of a bumpy road through the rest of this decade
hopefully not the entire rest of the decade,
but I'm also hopeful that we can get through that and on the other side of that
I do think this is the goal that we need,
just speaking for myself, to be able to do space flight in a way that is affordable and sustainable.
And that will sustain us then to do the great things that we want to do in the rest of this century.
Yes, sir.
>>AUDIENCE MEMBER: Yeah this ties into similar to that question
and this is more, I'm going to ask for your comments to this comment, observation actually.
It seems that we have kind of in this we've kind of created
this complexity in this world of limited resources in that we seem to be tending to test less
and satisfy ourselves that we're reviewing things sufficiently
and that we're adding more reviews to our processes.
And I know that this is a topic of discussion within the agency now, how things like class D projects,
which are considered low cost high risk projects, are being treated similarly from a review
standpoint as class A projects.
You wouldn't necessarily have a sounding rocket program
go through the same review process that JWST would,
but I so often see things like, we have these monthly management reviews,
which are ways for the technical people to tell management that everything is doing just fine
and the green font is also that emphasis that everything is just fine.
And the real issues in this,
and this is you know in line with the topic of today's seminar
that the real issues just don't seem to be brought to the surface
because there's that risk that because of our limited budget they're going to cancel us
or they're going to think that we're not doing our jobs right or something like that.
Can you just comment on that emphasis on reviews
and saying the right message versus actually taking that effort and doing more testing?
>>MR. CHAIKIN: Well I hope I'm not out of line by saying I would really like to hear
what Mike and Bryan and the others that come after me have to say about that,
because you're the guys that are on the front lines and again,
hard enough to put myself in the shoes of somebody during Apollo
trying to pull that off successfully, I can't imagine what it's like for a program manager today
who has tests it is come across his desk for approval
and out of ten tests that people want to have done to do it right, he can only afford to do five.
And he's got to triage which tests to do and which tests to throw out
because the fact of life is he's not going to get enough money to do all test tens,
knowing full well that failure means a black eye for the agency
and probably loss of job security for himself and others,
and yet that's the world that we're operating in today. So I respectfully punt. (Laughter).
Thank you.
Rogers: Thank you Andy
(APPLAUSE)