I’m currently reading Jim Collins’ new book, “How the Mighty Fall”, and it references how companies have to make difficult decisions with ambiguous data, and how hard it can be. He discusses the case of Iridium and Motorola, and how an early-stage experiment progressed to a huge bet, and how Motorola might have avoided the bankruptcy that followed. In the example, he discusses how hard it can be to make these decisions, since the data is rarely obvious to all at the time the decision is made.

To make his point, he references the book, “The Challenger Launch Decision”, by Diane Vaughan, and the events that led up to the decision to launch the Challenger in 1986. NASA had contracted a consultant regarding the cold temperature conditions (between 25-30 deg F) under which the Challenger would need to launch, and whether the O-rings might have an issue. Interestingly, the consultant gave the opinion that it may not be safe, due to the shuttle never having launched in such cold temperatures, and that the O-rings might fail and cause an explosion. The evidence cited was that the O-rings were often damaged at launches below 53 degrees. During a three hour meeting, NASA engineers and managers argued amongst themselves about what to do, since there were also O-ring failures at 70 degrees and above, and there wasn’t any clear evidence that a launch would be unsafe.

Interesting enough, it turns out that there WAS clear evidence available. It just wasn’t easy to visualize with NASA’s technology at the time. However, any IntraStage customer would tell you that what was needed was very easy to do in a tool like IntraStage. What they needed to map was a number of O-ring failures vs. launch temperature. If they had produced this graph, they would have seen that EVERY launch below 66 degrees showed O-ring failures, and this pattern mitigated substantially above 66 degrees. But, as Collins’ summarizes, “no one laid out the data in a clear and convincing visual manner, and the trend toward increased danger in colder temperatures remained obscured throughout the late-night teleconference debate.”

The O-ring Task Force stated, “We just didn’t have enough conclusive data to convince anyone.” But the evidence was there, hidden in the test data, and NASA didn’t have the tools to visualize it. The visualization would have saved the lives of seven people, including Christa McAuliffe (the first civilian astronaut).

That is the thing about good visualization tools. You never know exactly what report you’ll need, and you never know when you’ll need it. It might be an urgent teleconference where you are making a critical bet that, if wrong, could lead to a product recall or even loss of life. In these occasions, what you need are tools that allow you in real-time to deep dive on the test data, pivot on all kinds of scenarios, and visualize trends.