Daily Meh is an edited stream of consciousness, by Simen.

I’m going to respond to this post by Ryan Irvine, because I think it’s a sloppy critique of my post on correlation. Let’s see what he says:

In his Daily Meh piece that got a lot of attention the other week, Simen argues, with no shortage inscrutable post-Graduate circular logic, that “all we have to help us establish causal relationships is correlation,” or, in so many words, that orange production on one side of the world necessarily impacts the success of military coups on another.

I’ll simply note that Irvine can’t point to these alleged circularities in my arguments, or for some reason chooses not to in the rest of his response; and further, that the characterization of my argument “in so many words” is so skewed that I cannot imagine how anyone could have arrived at it without wilfully giving my post a very uncharitable reading. I stand by what I said, and people can go back and read what I originally wrote to see what I actually meant.

He continues:

My knee-jerk reaction to this argument was “What? No.” and I tried to argue as much over e-mail, and was, without much hesitation or explanation, told I was wrong in the roughly same terse manner that so many theoretical Computer Science professors had employed to tell me I was wrong in college.

I don’t know about those theoretical Computer Science professors, but I don’t feel the need to be apologetic about disagreeing on a philosophical issue. When someone sends me an email in response to something I wrote, I think it’s great! Even if they disagree. But if they tell me, “You are wrong, for the following reasons”, and I don’t find the following reasons convincing, I will respond in kind: “no, actually, I think you are wrong, for these reasons…” I don’t really see the need to be hesitant or give any further explanation than I’ve already given. I stand by what I wrote and the way I wrote it, but since this is private communication, I suppose I can’t give the whole audience a chance to see what it was all about. I suppose the above is meant to be insulting, but since there are a number of computer science professors I admire, I’ll take a comparison to them as praise.

Without further ado, I’ll do what I did in my email response to Ryan, namely, explain why I find his argument unconvincing. On to the meat of the critique, such as it is:

Listen, Simen is a smart dude—I like him so much I even stole his Tumblr theme!—but I see his argument as one dependent on repeatedly observed coincidences and leaps of faith, and a suggestion that just because we can’t prove that a relationship is not causal yet means that we should accept that it is for the time being. Though he has concocted the necessary verbal discrete math to support his argument—or at least otherwise confuse those who would disagree with him—this approach seems wholly unscientific to me, and it seems downright tired, perhaps because this kind of thinking has been continuously proved wrong since at least the 17th century.

I’ll let astrophysicist and generally awesome human being Niel deGrasse Tyson elaborate to that end. In this talk he gave about the role of Intelligent Design in science, Tyson explains that God has historically been and continues to be credited as the cause of pretty much anything that has ever occurred—even by scientists—at least until science comes up with a better explanation. Before Isaac Newton (co-)invented calculus to show why planets move the way they do, it was widely accepted that the elliptical movement of planets was simply God’s will. People didn’t care to give this problem much thought in part because they didn’t know how to approach it, and so they accepted God’s role (quite literally) on good faith. Years later, around the turn of the 19th century, French mathematician Pierre-Simon, marquis de Laplace used Newton’s work on planetary mechanics and gravitation to show why the universe is far more stable than previously thought. When asked by then emperor Napoleon Bonaparte why his work made no reference to God, Laplace replied, “I did not require him for this explanation.”

So God is the reason correlation doesn’t imply correlation? The above is an attempt to show that my thinking leads to absurd consequences, and so must be wrong. And it would indeed be absurd to think that the motion of the heavenly bodies was anchored in God’s will, not the laws of nature. The missing link in this counter-argument is the most important piece of all: the one that connects my claim, correlation implies causation (with the appropriate caveats), to the use of God as an explanatory factor in everything from astronomy to the social sciences.

Now, let me ask you: did all those theologians and natural philosophers who tried to “explain” this or that in terms of God’s will base their claims on an observed correlation between God and the phenomena they were trying to explain? I admit, if I saw a giant man in the sky swooshing his hands hither and thither, and I observed that the stars and the planets followed the movements of the Divine Hands, I would be inclined to believe that God was really causing their motion. As far as I’m aware, that has never happened. No one’s ever seen God doing anything, much less orchestrating the motion of the planets. There is no correlation to base this claim on. The theologians’ assumption wasn’t based on evidence. They didn’t observe anything that could only be God; rather, they were unable to imagine how it could not be God, and so didn’t bother to even look for alternative explanations. Which is something I made clear in my original essay: correlation alone is useless. You must not simply observe a correlation, you must also rule out alternative explanations. The theologians and scientists so deeply steeped in theism that they could not see anything but God as an explanation of the motions of the heavens, made two mistakes: one, they didn’t have any observed correlation between God and planetary movements in the first place, and two, they didn’t rule out alternative explanations.

“I like to take things all the way to their logical conclusions,” Simen writes, “and unfortunately, the absence of inferences based on correlation would mean that we could never know that one thing caused another.” Yeah, maybe if you’re really impatient.

I wonder how Ryan Irvine thinks classical mechanics was developed, or how it was verified. I’m really curious as to what alternative means of verification these proto-modern scientists had, means that are now lost to modern scientists, to verify a connection without observing it to happen consistently. How did Newton et al figure out gravity, and how did they verify that their mathematics had any connection to reality? Well, of course they observed that the equations always held true. Apply this force on this object under these circumstances, and that always happens (where that is predicted by Newtonian mechanics). Like it or not, this is how science is done. Ryan suggests that correlation is only for the impatient, but that implies there must be some more certain way of knowing causal relations, one that’s available to us if only we’re patient enough to sit around and wait for it.

That sounds like mysticism to me. That isn’t how science works. Every one of the relationships between cause and effect that science has allowed us to gain knowledge of, has been verified to hold through the mechanisms I described in my original essay: consistent correlation, and reduction to simpler, more “certain” causal relationships. These “certain” relationships in turn could only have been confirmed the same way, through consistent correlations.

Now, as for the role of faith. Irvine thinks that I’m all for blind faith in spurious correlations. On the contrary, the method I describe depends on the faith in only one proposition: that the future will resemble the past. That’s it. Given the belief that the future resembles the past, a belief we must take on faith (lest we argue in circles), a belief that I think every single human being holds, we can predict the future by looking at the constants of the past. The scientific establishment has refined its methods for weeding out coincidental correlations and confirming true correlations over centuries. This is a complicated process: you need repeatable and repeated experiments; you need to fiddle with the parameters one at a time to rule out other possible explanations; you need to make sure your results are consistent with established science, or else amass such convincing evidence that it would be a greater miracle for this new result to be false. The process is complicated in the abstract and even more so in practice. Of course you aren’t going to do full justice to it in the space of a few hundred or a few thousand words, or even in a few hundred thousand words. But even so, I contend that at the heart of this process is the observation of correlations, the testing of them and the ruling out of spurious correlations. I challenge Ryan and everyone else to scientifically prove a causal relationship without relying on correlations, and I’m sure that if they can do it, Nobel prizes are lined up for them.

Correlation implies Causation

An often repeated mantra is this: correlation does not imply causation. This is wrong.

In An Enquiry Concerning Human Understanding, David Hume lays out his famous argument against induction. In particular, he writes:

It is confessed, that the utmost effort of human reason is to reduce the principles, productive of natural phenomena, to a greater simplicity, and to resolve the many particular effects into a few general causes, by means of reasonings from analogy, experience, and observation. But as to the causes of these general causes, we should in vain attempt their discovery; nor shall we ever be able to satisfy ourselves, by any particular explication of them. These ultimate springs and principles are totally shut up from human curiosity and enquiry. Elasticity, gravity, cohesion of parts, communication of motion by impulse; these are probably the ultimate causes and principles which we shall ever discover in nature; and we may esteem ourselves sufficiently happy, if, by accurate enquiry and reasoning, we can trace up the particular phenomena to, or near to, these general principles. The most perfect philosophy of the natural kind only staves off our ignorance a little longer: as perhaps the most perfect philosophy of the moral or metaphysical kind serves only to discover larger portions of it. Thus the observation of human blindness and weakness is the result of all philosophy, and meets us at every turn, in spite of our endeavours to elude or avoid it.

He rightly observes that our best attempts to make sense of the world consist in reducing causes and effects to simpler parts, thus staving off our ignorance for a while, but what we’re doing is ultimately only pushing our ignorance onto ever simpler reactions. These elementary cause-effect relationships, whether they be “elasticity, gravity, cohesion of parts, communication of motion by impulse” or Quantum Mechanics, are still assumptions that we make. We don’t make them arbitrarily, but we still assume them, because no one has ever observed a cause. The question is, how do you establish causation if not by correlation? How do you come to know that one billiard ball causes another to move in a particular direction, if not by observing that the correlation holds at all times? Using modern physics, you may be able to explain it in terms of the motions and interactions of elementary particles; but how do you know that one particular elementary force or particle caused some other state of affairs to obtain? How do you know that photon caused that electron to make a quantum leap? (Fun fact: in common parlance, “quantum leap” means “huge leap”, while in physics, it is one of the smallest leaps you can possibly make, between two discrete energy states of an electron.) Ultimately, you know it because you observed the correlation and you ruled out other explanations.

Rod Knowlton writes:

Correlation alone can be a reason to look into something, but cannot be the basis for a conclusion.

I don’t want to say that he’s wrong, because I understand perfectly well what he means, which is that just because you observe a correlation between two kinds of events, that doesn’t mean there is a causal relationship between the two. It’s a perfectly useful phrase for describing that fact. The vast majority of people who hear that “correlation does not imply causation” will take from it that useful lesson without getting into serious philosophical territory. But I like to take things all the way to their logical conclusions, and unfortunately, the absence of inferences based on correlation would mean that we could never know that one thing caused another.

Consider a case that practically begs for the correlation doesn’t imply causation mantra. Say someone observes that every time there’s a military coup in Latin America, there’s a great year for oranges in China; there are twice as many ripe oranges in a given year if there was a Latin American military coup that year. There is no logical reason why orange harvests in China and military coups in Latin America would be related, so naturally, we’d be skeptical that there was a causal relationship between the two; even if we could clearly point out that one of them always preceded the other, that would hardly prove that one event was the cause of the other. How would we go about investigating this?

We’d do it by varying parameters. By looking at different conditions, we can observe whether or not the correlation always holds. If we have exhausted every possible cause, and found that the only thing that always precedes a good orange year in China is a military coup in Latin America, so that there is never a good orange year without a preceding coup, never a coup without a following good orange year, and no other factor that also consistently correlates with the orange year, then we can tentatively conclude that the military coup is somehow causing the good orange year. Since we’re unable to reduce this cause-effect relationship to a combination of simpler operations that we already trust, we may still be skeptical and quite open to the possibility that we’ve simply missed something — that there’s something else that also correlates, that we didn’t see. For all we know, there could even be ten different causes for ten different good orange years, and the correlation is simply accidental, but as long as we assume that the future resembles the past (a claim we cannot ultimately prove either, as Hume also reminds us), that’s exceedingly unlikely and becoming unlikelier every time we observe the correlation to hold.

But what if we could find some plausible combination of already accepted causal relationships that together add up to make the military coup affect the Chinese orange growth? This interaction here, this interaction there, and together, we get this? We may have guessed that the processes in the motor of a car makes it move, because we see that every time we start the motor and perform the necessary steps to control it, the car moves; but we can be sure because every process that goes on in a car’s motor are well-understood (not necessarily by you and me, not even necessarily by a skilled mechanic, but certainly by the aggregate of human knowledge) and we can see how they add up to the combined effect: the car moves. If we could find a chain of well-understood and trusted causal relationships that together forge a connection between cause and effect, military coup and oranges, we would have to accept the relationship as a truly causal one, and not just accidental correlation.

All this is just what Hume more concisely says in the quote above: we unload our uncertainty about causal relations on simpler, more fundamental causal relationships, but this can only stave off our uncertainty for so long. At some point, we have to ask: but how do we know that this elementary interaction causes that one? How do we know that the fundamental cause-effect relationships that we take for granted aren’t accidental? The answer is: we observe correlations! We know this because we have observed the correlation to hold many times, and by varying circumstances we can see that no other correlation holds as well; thus, to find out the cause of something that happened, we look for the strongest correlation, the one that holds most consistently. That is all. We cannot see, touch, smell, hear or taste a cause. Causes cannot be observed directly. The only thing we can actually observe is correlation, and the only way to protect ourselves from accidental correlations is to keep varying circumstances until only one correlation remains. Any number of things could occur before an eight ball rolls over to the right hole; by observing what happens in a variety of circumstances, if we observe that the only thing that always happens before the ball starts rolling is that another ball strikes it, we conclude that the cause of one ball’s movement is the force of another that strikes it. All we can ever do is reduce events to Hume’s “few general causes”, and regardless of the nature of these causes, our trust in them comes only from observing strong correlations.

For this reason, correlation implies causation. In fact, correlation is ultimately the only thing that implies causation. I was deliberately provocative when I said that “correlation does not imply causation” was wrong, because in the sense that “any correlation doesn’t imply causation”, it’s obviously true. I did this to highlight this interesting and possibly troubling epistemological fact (fact about how we come to know things): that the only way to establish knowledge of cause and effect is to observe correlations, because causes aren’t directly observable. Not any kind of correlation, of course. We must rule out alternative explanations. That’s actually a good nutshell version of the scientific method, which for understandable reasons is usually phrased differently: observe correlations of the form “A happens, then B”. Repeat experiement to confirm. Systematically vary circumstances until you start seeing correlations disappear. Continue until you have found one correlation that seems to stubbornly refuse going away, no matter what wacky edge-cases you throw at it. You may now, provisionally, conclude that A causes B. Wash, rinse, repeat.

Why is this troubling? Well, for one thing, we may fall prey to false causes, precisely the kind of thing the “correlation doesn’t imply causation” mantra warns against. No matter how many times one thing happens (or doesn’t happen) after another, we cannot be absolutely certain that it will (or won’t) happen again in the future. I’m reminded of the chemistry lab that exploded in Florida in 2007. They had done the chemical reaction that blew up the lab 175 times before without incident; then, suddenly, something went wrong and the lab went boom and real, actual people died. That particular correlation (between doing the reaction and a safe outcome) wasn’t true. Turns out, luck was what had been keeping them safe all this time.

Not only false positives (accidental correlations), but also false negatives (causes that don’t correlate) could trip us up because of our reliance on correlations. Maybe an accidental correlation is masking the fact that there are different causes to the same phenomenon; if none of the causes are repeatable, because of unique circumstances that we cannot duplicate, then we may never be able to comfirm that one time, it really was the dog that farted and not your sorry, excuse-making ass. If God’s invisible hand caused something to happen, that event might not be replicable, and we may never be able to confirm such miracles. And how the hell do you disambiguate between perfect correlations? If both A and B always happen before C, but nothing else is constant every time C happens, how the hell do we know which is the cause? How do we know it wasn’t the Invisible Pink Unicorn’s Invisible Pink Horn that caused C? We may never know, because all we have to help us establish causal relationships is correlation.