TK: find all links in here and create cross-references!
Objective probability is like phlogiston: it might seem real, it may even offer
some explanatory power, but in the end it’s a model with limited applicability.
Probability is a subjective judgment. In this admission lies a theory of
probability that is more objective than any idea of probability that sets out
with the goal of being objective.
Critically, the probability of an event is not:
- frequency;
- long-run tendency;
- based on symmetry of devices (like coins or dice); nor
- objectively fixed for multiple instances of the same event.
Although these ideas can be helpful to determine the probability of an outcome,
they do not define the probability in question.
This article is a short summary of the first handful of chapters of de Finetti’s
classic treatise on the fundamentals of probability1 Theory of Probability;
de Finetti; Wiley; 1974.. The book itself contains much more interesting
detail, proofs, edge cases, and arguments. Read it if this is your thing! The
quotes in this article are from that book, but I have taken the liberty of
editing them for conciseness. I have strived very hard to retain the
spirit2 Including the appropriate level of whimsy. of the message.
For any of this discussion to be meaningful, we need to constrain ourselves to
well defined events. By event, we mean something very general: something that
may or may not have happened, or that will or won’t happen; a statement that is
either true or false. Here are examples of events:
- The flipped coin lands heads-up.
- A banana is nutritious.
- Bob wins the lottery.
These events are not well defined. Here are some questions we may have to answer
before we can judge whether the events are true or false:
- Which flipped coin?
- What does nutritious mean? And which banana are we talking about?
- Is this referring to Bob winning the lottery next week, or the next time he
plays, or any time in the next year? What if he plays with a group that share
their winnings, does that count as Bob winning even if it technically wasn’t
his numbers?3 This question is from de Finetti. I found it really
insightful in terms of the weird edge cases you can encounter.
Note that events can also be combinations of other events, like “two dice are
showing a sum less than six”, which is really a combination of two die
throws4 The two throws are usually inpedendent, but if there’s any doubt
about it, that also needs to be specified for the event to be well defined..
“Emily Brontë was born 1954” is also an event that may or may not have occurred
in the past5 It did not. Brontë was born well before the 1900s.
This might all sound obvious and mundane, but it’s really important and people
suck at it. Here are some events people on the web have submitted when asked for
predictions for 2023. These are not well defined, making them meaningless for
the discussion to follow. I will point out one ambiguity in each, but I’m sure
you can find more.
ai will keep wowing us, but in 2023 actual change will be surprisingly little.
Besides this being a statement about how people will feel (“wow” and
“surprised”) without specifying which people will feel that way, it uses a
classic weasel: “actual” change. Whatever change occurs, the person submitting
this prediction can claim it isn’t an actual change.
Tech layoffs in the Bay Area intensify. When hiring begins again a large number
of the new hires are either remote or are in other geographical areas where
labour costs are lower.
By varying what “large number” means, we can always say that a large number of new
hires are in geographical areas cheaper than the Bay Area – this is particularly
easy because the Bay Area is notoriously expensive.
Tesla struggles to compete against old school manufacturers that have a grip on
quality and make advances on the ev tech side.
There are so many potential meanings of the phrase “struggle to compete” that
it’s always going to be true of anyone in any situation. (Also note the “old
school manufacturers that have a grip on quality” weasel – if Tesla competes
well against some old school manufacturers, the person submitting this can say
that those particular manufacturers “don’t have a grip on quality” and thus be
right again!)
Operational definitions make for well defined events
One way out of the problem with under-specified events is to adopt operational
definitions. An operational definition is a series of steps we can execute, and
the last step results in the value we are looking for.
For example, the event “Next week’s lecture will be attended by over 130 people”
isn’t well specified.6 Thanks, Deming, for this example. Some questions you
might ask:
- Do you count staff present, or just audience?
- If one audience member has brought their toddler, who has no interest in the
lecture itself, does that count? - How about the janitor that happened to be slowly sweeping the floors of the
lecture hall during the lecture because they had an interest for the subject? - What about the student that were supposed to attend did not have an interest
and left halfway through?
How you count something affects what number you get. There is no true number of
people attending the lecture, there are just various procedures for achieving a
count, and the selection is subjective!
There are many ways to operationalise the count of lecture attendees:
- Stand in the door of the lecture hall and survey everyone entering if their
purpose for being there is attending the lecture. - Take a photo from the front of the hall at the 40 minute mark, and count the
number of heads visible in it. - Feel each seat two minutes after the lecture has ended and check how many of
them are still warm.7 But then you need a protocol for how to handle the
case when the seat is still occupied!
If the event in question is about a fact where at least one interpretation may
be well-known and widely accepted, it is helpful when operationalising it to
have a procedure that involves consulting reliable sources of news or
statistics, e.g. Bloomberg, English Wikipedia, industry statistics
organisations, the various Census Bureaus of low-corruption countries, and so
on.
Now that we know something about what we are going to talk about (well defined
events), we can finally start talking about it.
Logic is the language of the definitive. Logic is objective. In logical
reasoning, events are either certain or impossible. If we agree on an
implication, e.g. “When it rains, the lawn is wet”, then once we observe rain,
the lawn is certainly wet. On the contrary, if we know the lawn is dry, rain is
impossible.
If we are ever in doubt about a logical statement, we can look at the real-world
outcome of the event, and from this determine whether the logical statement
about it was correct. There is nothing subjective about this process: we apply
the operational definition, and the result tells us whether logical statement
was either true or false. It doesn’t matter who does this, because everyone
would have gotten the same result.
A lot of statistically un-trained people are stuck in this framework of logic,
where things are certain and causes follow a logical progression. They only know
the language of logic, so they express predictions about the future (which are
by nature uncertain) in terms of logical reasoning (which deals specifically
with the things that are certain).
It is common to try to “guess”, among the possible alternatives, the one that
will occur. This is an attempt often made by experts who are inclined to precast
the future in the forge of their fantasies. Everyone will no doubt have noticed
how often the “foresights of experts” turn out to be completely different from
the facts, sometimes spectacularly so. In the main, this is precisely because
they are intended as guesses which “deduce”, more or less logically, a long
chain of consequences – still considered necessarily plausible – from the
assumed plausibility of an initial hypothesis.Here also one might note that the hypothetical reconstructions of historical
events made by scholars and novelists, based on scanty data, are also guesses in
the above sense. Ineptitude or laziness prevents us from seeing how many other
possibilities there are, besides the first one we happened to think of.
In effect, the aforementioned experts have a perspective of the world where
events uniquely and certainly are caused by earlier events in a logical chain.
With this perspective, once you know one thing for certain, you can extrapolate
as far into the future as you want, and arrive at definitive guesses at what
will happen in the future.
This is the same sort of flaw as saying, “When a passenger rides the train they
have a ticket.” That’s a plausible consequence, but not a certain one, because
sometimes they do not. All we can do, while remaining in the realm of objective
logic, is to say, “When a passenger rides the train it is possible they have a
ticket.”
When logic is insufficient to tell us whether something is certain or
impossible, all it allows us to do is say that it is possible. Everything we
are uncertain about goes into the “possible” bucket. This feels a bit
unsatisfactory. After all, it’s also possible a person on the street has a
ticket, but it seems somehow… more possible for the passenger on the train.
The thing we are not content with, is the agnostic and undifferentiated attitude
towards all those things which, not being known to us with certainty, are simply
“possible”. In logic, there are no degrees of possibility: it is possible
(equally possible) that it snows on a winter or summer day; that a great
champion or novice wins the competition; that every student, whether
well-prepared or not, will pass an examination; that next Christmas you will
find yourself at any place in the world.However, we do not content ourselves with this, and, in fact, it is not our real
attitude. Faced with uncertainty, we feel a more or less strong propensity to
expect that certain alternatives rather than others will turn out to be true.
This feeling is what we tap into when we measure varying degrees of possibility.
That leads to probability: expressing the level of confidence we have in things
that are possible, yet uncertain to us.
Probability is based on the feeling that some things are more likely than
others. As with any feeling, it is necessarily subjective. If I feel that
there’s a 20 % chance one of our common acquantiances need to go to the hospital
in the next six months, and you feel there’s a 70 % chance, there’s no objective
measure by which we can
