Here’s the Cable Guy Anomaly: you’re waiting for a cable guy to come make an installation, and he has a 75% reliability, meaning a 75% chance he’ll show up at all. If he does show up, it’ll be sometime between noon and 2pm, with equal likelihood in each ten minute interval. As the minutes pass in vain, there are two probabilities which vary through time: the probability he’ll show up at all, and the probability he’ll show up within the next ten minutes. The “anomaly” is that the former goes down the longer you wait, but the latter goes up. Actually, there’s nothing anomalous about this at all, it agrees perfectly with the analysis if you do it right. But it strikes many people as enigmatic because it seems counter-intuitive.
At exactly noon, the odds the Cable Guy will come at all, are 75%. As 1:00pm rolls around with no show, those odds should be smaller. There are certain worlds where the technician shows up, and certain where he doesn’t. Of the former, some have him showing up before 1pm. But he hasn’t, so those possible universes have been ruled out, while all the no-show universes remain. It turns out that, if he hasn’t shown up by 1pm, the odds he’ll show up at all have gone down to 60% (I’ll prove this below). But at noon, the odds of a showing in the next ten minutes are a miniscule 6.25% (75% divided by twelve ten-minute intervals). If he hasn’t come by 1pm, the odds of a showing in the next ten minutes go up, to 10% (again, I’ll prove it later).
TL;DR: Odds he’ll show up at all: These decrease. Odds he’ll show up in the next 10 minutes: These increase.
This sort of situation is the subject of a paper I recently read: “The Ups and Downs of the Hope Function In a Fruitless Search”, by Ruma Falk, Abigail Lipson, and Clifford Konold, from the book Subjective Probability (edited by Wright and Ayton). Here’s the Amazon Link
Waiting hopefully for the cable guy is isomorphic to searching hopefully for an item. For example, maybe there’s a 75% chance you own a certain book, and your bookshelf has twelve sections. Searching for the book, section by section, is identical to waiting for the cable guy, 10-minute by 10-minute. At first, before you do any searching, there’s a 75% probability the book’s there, and a 6.25% chance you’ll find it in the first section. By the time you’ve searched half the sections fruitlessly, the odds you’ve got the book at all go down to only 60%, whereas the odds you’ll find in the next section go up to 10%.
These probabilities are actually extremely easy to calculate– you can do it in your head– if you know how. I’ll explain the trick below. The authors of the original paper weren’t aware of the trick themselves, initially. They (the authors) were interested in studying peoples’ intuition about this phenomenon, and gave surveys to high school and college students asking questions similar to the ones above, along with a space for the surveyed students to explain their answers. The study supported my calling the phenomenon an “anomaly”: practically nobody had the intuition right! There was only one kid who nailed the questions, a mystery whiz kid identified only as “Subject No. 62″, who not only got the right answers but “explained” them with a brilliant trick, teaching the survey-writers something new!
The Easy Case: Initial Probability 100%
Before I can explain the whiz-kid’s trick, we must study the “easy case”, which is when the initial probability is 100%.
Suppose you’re 100% certain the cable guy will come sometime between noon and 2pm, with equal likelihood in each 10-minute interval. Initially, before doing any waiting, the odds he’ll show up at all are 100%, and the odds he’ll show up in the first 10-minute interval are 1/12, or about 8.33%. If 1pm rolls around with no Cable Guy, then the odds he’ll show up at all are still 100%. The odds of him showing up between 1pm and 1:10? Well, there are six 10-minute intervals left, and we know the guy will show up in one of them, with equal chance each, so it’s 1/6 odds our cable will be installed within the next 10 minutes.
These are calculations you can do in your head. If you’ve waited 30 minutes with no Cable Guy (in the above example), what’s the chance he’ll come in the next 10 minutes? There were originally twelve 10-minute intervals. Substract three and there are nine remaining. Chances are 1/9 you’ll meet your cableman in the next 10 minutes. This is why it’s called “the easy case”. What Subject No. 62 figured out was how to transform the general question into the easy case.
The Trick: Reducing to the Easy Case
Here’s (part of) the question which inspired the whiz kid. You have a desk with eight drawers. There’s an 80% chance the desk contains an important letter. Suppose you search four drawers, finding nothing. What are the new odds the desk contains the letter? What are the odds you’ll find the letter in the fifth drawer?
The brilliant idea is, extend the desk by adding two new locked drawers, and pretend the desk has a 100% chance of containing the letter– but it might be one of the locked drawers, which you can’t open. Instead of asking “is the letter in the desk”, the new question is “is the letter in one of the unlocked drawers?” The question suddenly becomes much easier because the initial probability is 100%. After fruitlessly searching four drawers, six drawers remain, of which four are unlocked, so the chances the letter’s in an unlocked drawer are 4/6 or about 67%. This translates back to the original question: after a failed search of four drawers, the desk is 67% likely to contain the letter at all. Similarly, in the transformed question there are six drawers left and the fifth drawer is one of them, so there’s a 1/6 chance of finding the letter in the fifth drawer.
Why add 2 drawers, why not 3? With two locked drawers, the original probability that the letter is in the desk at all (80%) ends up being equal to the chance, in the transformed situation, that the letter is in an unlocked drawer (8/10=80%). In this example, it’s easy to figure out that 2 is the magic number of locked drawers to add. You can also deduce a general formula.
Suppose event E has initial probability P of occurring, and will occur with equal likelihood in one of N places, if at all. We’d like to add X new “locked” places, such that, if there’s a 100% probability of an event happening in one of the N+X places (with equal likelihood for each), then the probability the event happens in one of the N “unlocked” places is P. In other words, N/(N+X)=P. This solves to: X=(N/P)-N. In the above example, we had P=.8 and N=8, so X=(8/.8)-8=2, so we 2 locked drawers was the correct number to add. We didn’t really need to use the formula to find that, though, and in many cases, the formula isn’t necessary.
Let’s do the Cable Guy example from the opening paragraph. Suppose there’s a 75% chance the Cable Guy will come. If he comes, he’ll come between noon and 2, with equal likelihood in each of the twelve 10-minute intervals. Assuming he hasn’t come by 1pm, what’s the likelihood he’ll come at all? What’s the likelihood, in that case, he’ll show up by 1:10?
Here N=12 and P=75%=.75. Instead of “unlocked” 10-minute intervals, we’ll speak of “10-minute intervals when we’re at home”. To the twelve 10-minute “at-home” intervals, add four “locked” 10-minute intervals, or if you prefer, forty minutes “when you aren’t home”. I choose this number because 12 is 75% of 12+4. I didn’t really need the formula, but if I did use it, it’d give (12/.75)-12=4. In the transformed question, the Cable Guy has a 100% chance of showing up sometime between noon and 2:40pm, but you have to go to work at 2pm. Assuming Mr. Time-Warner hasn’t shown up by 1pm, what’s the likelihood he’ll come at all while we’re still home? Well, there are ten 10-minute intervals left until 2:40, but we’re only at home for six of them. So the answer is 6/10 or 60%. What’s the likelihood, given no show by 1pm, that the guy will show by 1:10? Again, there are ten 10-minute intervals left, and the interval from 1pm to 1:10 is exactly one of them, so the answer is 1/10 or 10%. See how easy it was to derive the answers in the opening paragraph?
Sometimes (N/P)-N ends up being fractional. What if Roadrunner was slightly less reliable, with only 70% reliability instead of 75%? Then we’d take P=.7, N=12, and get (12/.7)-12, or 36/7. Fractional, but it still ends up working. The odds of an appointment in the first 10 minutes? Well, there are 12+(36/7) total 10-minute intervals in the transformed question, of which the first 10 minutes are one, making 1/(12+(36/7)) or about 5.83%. What if you’re still waiting after 50 minutes, then what are the odds the appointment will go through at all? 7+(36/7) intervals remain in the transformed situation, of which you’ll be home for 7. So the answer is 7/(7+(36/7)) or about 58%.
Confusion about Probability
Besides the raw math– impressive enough on its own– the paper also had interesting results about peoples’ intuition about this math.
What I found most interesting was, the study provides evidence that people seem to reason as though probabilities were physical properties of matter. In the example with the desk with the eight drawers and an 80% chance a letter is in the desk, many people reasoned as though “80% chance-of-letter” was a fundamental property of the furniture, up there with properties like weight, mass, and density.
Many reasoned that the odds the desk has the letter, stay 80% throughout the fruitless search. Thus, they reasoned, it would still be 80%, even if they searched seven drawers and found no letter. And these were people with some education about probability! One problem is people were tending to overcompensate to avoid falling into the Gambler’s Fallacy. They were educated, well-learned people, and they knew that the probability of a fair coin falling heads remains 50%, no matter how many times in a row heads have already been rolled. They seemed to generalize this to the letter search. There’s an important difference, though: the coin flips are independent of each other. The drawer searches are not.
In a followup study, when the modified questions were posed, with two extra “locked” drawers and a 100% initial probability of a letter, miraculously the respondents’ answers showed dramatic improvement. Even though, formally, the exercises were isomorphic.
FURTHER READING
The Ubiquity of Near-Impossible Coincidences
Ways to be Better at Algebra
Applications of the Sorites Paradox
Nonstandard Worlds
How to Stand on the Shoulders of Giants