## Tuesday, July 9, 2013

### Physical Probability versus Evidential Probability

There is a very important divide between two different types of probabilities. These are:
(1) physical probability and
(2) evidential probability.
The nature of (2) evidential probability seems to be very important indeed to the issue of uncertainty in economic life.

For if one cannot be certain of something, especially of a particular event in the future, then one must face risk, uncertainty, or degrees of uncertainty.

I can’t claim to have any great expertise in mathematics or probability theory, so I have quite likely made mistakes below.

I discuss the types of probability, as follows.

(1) Physical probabilities (or objective or frequency probabilities)
These exist within a strict domain of certain random events, phenomena or processes, such as (non-fraudulent and fair) games of chance like roulette and rolling of dice, and the states of physical systems in both classical and quantum mechanics.

In any such phenomena, over a long period of time, any particular possible event or state of the system supposedly tends to occur at a persistent rate or “relative frequency.”

The current widely-used set of axioms for probability were formulated by Andrei Nikolaevich Kolmogorov in 1933.

Like me, I suspect most people are familiar with physical probability from standard courses on discrete mathematics. A real physical probability (or objective/frequency probability) is limited to strict conditions:
(1) it must be random, in the sense that, when an outcome occurs, it is from a set of possible known outcomes, and one such outcome is sure to occur and it is impossible to predict with certainty what that outcome will be. E.g., in tossing a fair coin, we know that the result will be heads or tails and can denote the set of outcomes by {heads, tails}. The latter is the sample space.

(2) it must have a sample space (or set of outcomes) that is known, in the sense that there are finitely many outcomes of the random process and these can be stated. (An event is defined as a subset of the sample space.)

(3) All events are equally likely to occur.
Simply stated, the probability P(E) of any event E in a finite sample space S, where all outcomes are equally likely, is the number of outcomes for E divided by the total number of outcomes in S.

But the philosophical interpretation of physical probabilities are disputed, and have been interpreted in the following ways:
(1) in the Classical interpretation (of Laplace);

(2) in a frequentist sense (as argued by Venn, Reichenbach and Richard von Mises) (also called aleatory probability), or

(3) in a propensity sense (as argued by Popper, Miller, Giere and Fetzer).
Note that Richard von Mises was the brother of Ludwig von Mises!

(2) Evidential probability (or Bayesian or subjectivist probability)
An evidential probability is distinct from a physical probability and can be assigned to any proposition, even when the process is not within the strict domain of physical probability. It is sometimes defined as a method of representing the subjective plausibility of an event and the degree to which a statement p can be supported by available evidence, or the degree of belief one can have that an outcome will happen.

An evidential probability is a type of conditional probability, denoted by the expression P(A, B), which is to be read as “the probability of A, given B.”

Examples of evidential probability statements could be:
(1) The extinction of the dinosaurs was caused by an asteroid hitting the earth.

(2) Napoleon did not die by poisoning.

(3) The rate of unemployment in the UK in 2014 will be 3%.
The precise nature of evidential probability is disputed, and there are four interpretations as follows:
(1) Classical interpretation (of Laplace);

(2) the subjective interpretation (de Finetti and Savage);

(3) the epistemic or inductive interpretation (Ramsey, Cox) and

(4) the logical interpretation (Keynes and Carnap).
There is a fundamental division between objective and subjective theories of probability, where subjective interpretations identify probabilities with degrees of belief of an individual, while objective theories see probabilities as indicative of the objective behaviour of the real world, such as relative frequencies or propensities.

The subjectivist view of evidential probability seems to be associated with Bayesianism or advocates of epistemic probability, who regard evidential probabilities as having a subjective status by which they measure the “degree of belief” of the individual assessing the probability of an event or outcome.

Precisely what position Keynes took in the Treatise on Probability on evidential probability may be disputed, but (for what it is worth) this is what Wikipedia says:
“One of the main points of disagreement lies in the relation between probability and belief. Logical probabilities are conceived (for example in Keynes’ Treatise on Probability) to be objective, logical relations between propositions (or sentences), and hence not to depend in any way upon belief. They are degrees of (partial) entailment, or degrees of logical consequence, not degrees of belief. (They do, nevertheless, dictate proper degrees of belief, ...) Frank P. Ramsey, on the other hand, was skeptical about the existence of such objective logical relations and argued that (evidential) probability is "the logic of partial belief" ("Truth and Probability", 1926, p. 157). In other words, Ramsey held that epistemic probabilities simply are degrees of rational belief, rather than being logical relations that merely constrain degrees of rational belief.”
http://en.wikipedia.org/wiki/Probability_interpretations
For example, the truth of many propositions inferred from a body of evidence is not strictly necessary (or entailed), but merely probable. We call these types of arguments inductive reasoning. Keynes may well have called this inductive argument “partial entailment.”

Keynes’s logical interpretation of evidential probability takes the latter to be a measure of the logical relation between a proposition and the evidence adduced for it, but this relation is not deductive (that is, an inference is not strictly entailed by evidence). The logical interpretation of evidential probability also seems to be an objectivist one. In contrast, Bayesianism is a subjectivist view of evidential probability, and probabilities are contingent and not logical.

Today almost all philosophers reject the logical interpretation of probability as flawed (Lyon 2010: 112)

Conclusion
It should be apparent that physical probability is highly restricted in its usefulness to special domains, and useless for many real world assessments of probability.

While Bayesianism seems to be an important modern theory of evidential probability, a more interesting question is: do economic agents, say, the average consumer or capitalist making an investment decision, all or generally behave as required by Bayesian probability theory?

If not, it follows that Bayesian probability has far less relevance to the theory of decision making in economic life than imagined in neoclassical economics.

Furthermore, even if some agents do use Bayesian probability, are their probability calculations really providing useful measures of the probability of future events affecting economies?

Other Questions
Some questions that I do not have answers to:
(1) In probability theory, there are (1) “absolute probabilities” (also known as “unconditional probabilities”) and (2) “conditional probabilities.”

Are “absolute probabilities” to be identified with “physical probabilities,” and “conditional probabilities” to be identified with “evidential probability”?

(2) Some thinkers (such as Alfred Rényi and Popper) have developed a theory of probability that takes “conditional probabilities” as the primitive notion. Has there been any resolution of this in modern theory?

(3) Did Keynes change his mind on probability theory after he read Frank Ramsay’s the critique of A Treatise on Probability?

(4) Is the model of decision making by the rational agent in neoclassical economics basically derived from Frank Ramsay’s Bayesian and subjectivist theory probability?
BIBLIOGRAPHY
Lyon, A. 2010. “Philosophy of Probability,” in F. Allhoff (ed.), Philosophies of the Sciences: A Guide. Wiley-Blackwell, Chichester, UK and Malden, MA. 92–127.

1. "Is the model of decision making by the rational agent in neoclassical economics basically derived from Frank Ramsay’s Bayesian and subjectivist theory probability?"

So far as I know the answer is yes. The decision maker in neoclassical economics, when making decisions under conditions of what neoclassicals call "uncertainty", operates in a world of what is called "expected utility". This utility is calculated in a sort of gambling manner based on probabilities.

So, for example, is it in the consumer's utility function to take out home insurance given that there is a 10% risk of the home burning down? The consumer is imagined to have certain preferences (risk averse, risk neutral etc.) and then to weigh these preferences against the probabilities. (Which, of course, are known which is why a Post-Keynesian would say that the consumer does not exist in an uncertain world at all...).

The expected utility theory is pulled from Morgenstern and Von Neumann's original paper on game theory. But some also trace it back to Ramsey. Regardless the theory is indeed Bayesian:

http://en.wikipedia.org/wiki/Bayesian_probability#Personal_probabilities_and_objective_methods_for_constructing_priors

However, we really do not need a critique of Bayesian theory to critique expected utility theory (although it helps). Instead all we have to do is go back to the original paper by Morgenstern and Von Neumann and note that their theory only works if the "bet" is repeated over and over again. In the expected utility theory in neoclassical micro this is not taken into account and so the whole thing doesn't hold up. Steve Keen points this out in his book if you're interested in the source.

1. Thanks for this comment. Confirms quite a few things. Must follow up the Keen reference.

cheers

2. I should just clarify that my impression of the neoclassical textbooks is that they have a weird dichotomy here. They seem to consider only stuff that is bought in a risk environment -- so, financial assets and insurance contracts and the like -- to be subject to expected utility and hence all of what I discuss in the comment. However, as Keynesians well know, all sorts of things are subject to uncertainty (leaving aside that we disagree with them on what that term means).

So, money is seen by Keynesian economics as a hedge against the future, but it is certainly not discussed (so far as I know) in expected utility theory (EUT) like that. It's just a numeraire. But, given that money might have a risk/uncertainty aspect, and probably other commodities too (recall Sraffa's "own rate" discussion), we could probably apply EUT across the board.

Do the neoclassicals do this? I would say: no. But the potential is there. So, to answer your original question as to whether neoclassical theory uses this probability stuff: yes, but only in part. The more interesting question is: if it were consistent would it fully utilise this framework? I would say: absolutely!

I'll take my Nobel Prize now... ;-)

3. "Instead all we have to do is go back to the original paper by Morgenstern and Von Neumann and note that their theory only works if the "bet" is repeated over and over again."

Uhm, no. Von Neumann and Morgenstern merely invoke frequentist interpretation of probability, but do not claim that their theory is relevant only for repeated gambles (since there would be no uncertainty then, that would make no sense). In fact they quite explicitly state that their framework is static ("However, it would be an unnecessary complication [...] to get entangled with the problems of the preferences between events in different periods of the future", Theory of Games and Economic Behavior, section 3.3). Moreover, expected utility can be formalized without any reference to frequentist probability (as done e.g. by Savage), so that probabilities just express subjective beliefs.

"So, money is seen by Keynesian economics as a hedge against the future, but it is certainly not discussed (so far as I know) in expected utility theory (EUT) like that."

There is a literature on models with incomplete markets, where demand for money comes from agents trying to accumulate assets to self-insure against bad individual shocks. See e.g. section 4.2 here for some references: http://scholar.princeton.edu/markus/files/survey_macrofinance.pdf

4. @ivansml

The original VN and M paper is not, as far as I've read, to do with decision-making under uncertainty. It is to do with reformulating utility theory to use probabilities rather than what is currently used. That is why they use repeated gambles -- because the theory is not constructed to counteract uncertainty, but to reformulate utility theory because they consider it inadequate.

It was later that neoclassicals used their framework to deal with uncertainty. And, for the record, I believe that M endorsed this. Not to say, of course, that his word is gospel because expected utility theory is one of the stupidest things that neoclassicals have ever invented.

On the money point, I'm not surprised. Hahn was doing something similar with general equilibrium models in the 1980s.

2. On the question if and to what extent KeynesI did change his view on probability in response to Ramsey's critique, I would like to say that (the argumentation is more fully presented in my book "John Maynard Keynes" (in Swedish) it has been exceedingly difficult to present evidence for Keynes really fundamentally changing his view on probability. Ramsey's critique was mainly that the kind of probability relations that Keynes was speaking of in his Treatise actually didn't exist and that Ramsey's own procedure  (betting) made it much easier to find out the "degrees of belief" people were having. This is questionable from both a descriptive and a normative point of view. What Keynes is saying in his response to Ramsey is only that Ramsey "is right" in that people's "degrees of belief" basically emanates in human nature rather than in formal logic. Ramsey's critique made Keynes more strongly emphasize the individuals' own views as the basis for probability calculations, and less stress that their beliefs were rational. But Keynes's theory doesn't stand or fall with his view on the basis for our "degrees of belief" as logical. The core of his theory - when and how we are able to measure and compare different probabilities - he doesn't change. Unlike Ramsey he wasn't at all sure that probabilities always were one-dimensional, measurable, quantifiable or even comparable entities.

3. 1) Conditional probabilities could be defined using what you call frequency probabilities/Kolmogorov's axioms of probability. P(AB)=P(A)*P(B|A)=P(B)*P(A|B)
As long as events are related (dunno if I'm maybe mistranslating the term) you have your conditional probabilities.

You are also fudging the necessary conditions for the existence of frequency probabilities. Sample space could be infinite (example: tossing a coin till heads). Probabilities of the elementary events need not be equal (i.e. dices could be loaded). Also I'd like to add that probabilities (in a sense that is used by the probability theory) only apply to non-unique events. Period.

If you want to learn more about probability theory, it's an easy subject at its core. It will take maybe a day to learn the basics.

4. (1) In probability theory, there are (1) “absolute probabilities” (also known as “unconditional probabilities”) and (2) “conditional probabilities.”

Are “absolute probabilities” to be identified with “physical probabilities,” and “conditional probabilities” to be identified with “evidential probability”?

While I'm not a mathematician specialising in probability theory, I believe you are correct on this point and that there are different philosophical interpretations of probability theory.

(2) Some thinkers (such as Alfred Rényi and Popper) have developed a theory of probability that takes “conditional probabilities” as the primitive notion. Has there been any resolution of this in modern theory?

I can't say for sure. But I am aware that George Boole did discuss conditional probability in his writings. You may wish to read An Investigation of the Laws of Thought.

(3) Did Keynes change his mind on probability theory after he read Frank Ramsay’s the critique of A Treatise on Probability?

Please see this recently-published article in History of Economic Ideas as an answer to that question.

http://www.torrossa.it/resources/an/2569189

(4) Is the model of decision making by the rational agent in neoclassical economics basically derived from Frank Ramsay’s Bayesian and subjectivist theory probability?

Frank P. Ramsey is one of the three creators of Subjective Expected Utility decision theory, the standard decision theory used in many economic models to depict agents. The other two creators were the Italian mathematician Bruno de Finetti and the American mathematician Leonard J. Savage.

Daniel Ellsberg famously criticised the axioms of choice as posed by S.E.U. decision theory in a seminal article published in The Quarterly Journal of Economics in 1961.

http://qje.oxfordjournals.org/content/75/4/643.abstract

He would massively expand upon the arguments made in the article and turn it into a monograph submitted as his doctoral thesis. Although he finished it in 1962, it was only in 2001 that it was finally published.

http://www.amazon.com/Risk-Ambiguity-Decision-Studies-Philosophy/dp/0815340222

That stated Lord Keynes, I would also suggest taking a look at David J. Marsay's blog. He has made notes on probability theory that might be of interest to you.

http://djmarsay.wordpress.com/

5. (1) ...
Are “absolute probabilities” to be identified with “physical probabilities,” and “conditional probabilities” to be identified with “evidential probability”?

No. There are conditional probabilities which are "physical". Imagine consecutive random extractions from a deck of cards of known composition. The probability of an ace at the first extraction would be called a "marginal" (unconditional) probability of physical nature.
Now consider the probability of an ace at the third extraction, after that two non-aces have been selected and discarded from the deck. This conditional probability is also of physical nature.

(2) Some thinkers (such as Alfred Rényi and Popper) have developed a theory of probability that takes “conditional probabilities” as the primitive notion. Has there been any resolution of this in modern theory?

As you said, the logical approach is now considered false, thus I believe the major view is that unconditional probability simply does not exist and all probabilities are always conditional.
Kolmogorov and the text books simply explain the way in which a new evidence updates the probability space (PS). If you want to remain coherent with your initial PS, you have to discard the outcomes that have been disproved by the evidence and re-proportionate accordingly the remaining outcomes (so that the probs will still sum to one). That is why
P(A|B)=P(AB)/P(B)
This does not gives to P(B) any special status, in fact also B is conditional to anything assumed before. For example the mechanism of extraction of the card, which has to be so symmetric and calibrated all alonmg the experiment...
In other words, there is no tabula rasa. There is no standard (or logical) Probability Space that can be "assumed without assumptions".
P(B) is in fact a P(B|H) and the "conditional" is
p(A|BH)=P(AB|H)/P(B|H)
The exclusion of the H is just for notational convenience, as it should be carried away all the time.

Finally, a fundamental reference for Subjective and Bayesian is
De Finetti. de Finetti, B. (1974), Theory of Probability.