One of the best lines in modern cinema (my opinion) is spoken by James Earl Jones’ Admiral Greer character in The Hunt for Red October. He’s asked what can be deduced from all the soviet naval traffic in the Atlantic and responds, “The data supports no conclusions yet.”
This is a great line for any kind of analyst to remember. It should be repeated in the interrogative (“Does the data support any conclusions yet?”) any time an analyst (in any discipline) is about to suggest a course of action.
Data and conclusions are two ends of an investigative, evidentiary spectrum that (again, my opinion) few modern analysts know how to traverse. Data and conclusions are at two ends of the spectrum yet they’re not polarities of each other. Understanding how the two are linked is crucial to understanding what conclusions any data can reasonably support.
Consider the data to conclusion vector. Can the data support any conclusions other than the one being offered?
- If yes, what about the data makes this conclusion more certain than all the others?
- If not, excellent, the data supports this one, singular conclusion and a direct cause-effect relationship has been established. One can feel safe taking action based on said data because only one possibility will exist for any action taken.
The test of any such data to conclusion vector is the ability to reverse it with predictable results. That reverse is the conclusion to data vector, something most people know as predicting outcomes. If your data uniquely supports a conclusion, then you must be able to replicate the conditions that produced that conclusion and this time predict what the resulting data will be.
This, I know, is a direction few analysts are bold enough to tread.
Plausible Deniability versus Predictive Accuracy
It’s much safer business-wise to offer low grade, low accuracy predictions with little to no verifiable data backing up a decision than to offer high grade, high accuracy predictions with lots of data. The former even has a term associated with it, plausible deniability, as in “What? It didn’t work? Gosh that’s a shame. We told you it was the best advice we had based on the limited information we had at the time.”
The latter is not. In fact, an evidentiary trail for decision making is highly actionable (nice) and also highly vulnerable to litigation (ouch!). I touched on this briefly in The Unfulfilled Promise of Online Analytics, Part 1.
Let me give you an example. If I say “I weigh more than I should” this is an opinion and people can disagree but it isn’t open to litigation because it’s only an opinion and even if I’m an expert witness, there are other expert witnesses who may disagree that I weigh more than I should and so it goes.
However, if I say “I weigh 227#” then we’re open to litigation because someone could say “How did you calibrate that scale?”, “Are you certified to use that instrument to come up with that value?”, “Has that instrument been independently validated as an accurate device for taking such a measurement?” and it doesn’t matter how well things are documented, only that by offering something exact I’ve opened it to dispute.
I was once congratulated on receiving my patents and the products being derived from them. It amused me because this “verifiable data” vs “probable deniability” thing is also found in the patent process.
According to Todd Sullivan, NextStage’s IP Counsel, “In patent prosecution, you try to get good, broad claims allowed without making definitive statements about why the claims should be allowed. If the judge doesn’t see a clear explanation for why the claims were allowed, the judge will defer more to the patent examiner. Similarly, opposing counsel will have to work significantly harder to defeat the patent because no Achille’s heel will have been defined.
“In the end, it’s not about being right, it is about not being wrong. And it’s not about taking a chance to be successful, it is about advancing timidly and limiting vulnerability.”
(my deep thanks to Todd Sullivan of Hayes Soloway, P.C. Intellectual Property Worldwide for his help with this post)
Please contact NextStage for information regarding presentations and trainings on this and other topics.
Sign up for The NextStage Irregular, our very irregular, definitely frequency-wise and probably topic-wise newsletter.
You can follow me and my research on Twitter. I don’t twit often but when I do, it’s with gusto!
Have you read my latest book, Reading Virtual Minds Volume I: Science and History? It’s a whoppin’ good read.