Thursday, July 11, 2013

Probability and Uncertainty

There are two fundamental types of probability with subcategories:
(1) Physical/Objective probabilities (class probabilities), divided into:
(i.) A priori probabilities (mathematical/Classical probabilities)
(ii.) Relative frequency probabilities (or a posteriori/empirical/experimental probabilities), and
(2) Subjective probability (or evidential/Bayesian probability).
These are discussed below, with the issue of uncertainty in section (3).

(1) Physical/Objective Probabilities
Again, these are divided into:
(i.) A priori probabilities (mathematical/Classical probabilities), and
(ii.) Relative frequency probabilities (or a posteriori/empirical/experimental probabilities).
Objective probabilities are either in practice or in theory quantifiable with a numerical value (or numerical coefficient of probability). The numerical value that describes the likelihood of an occurrence or event can range from 0 (impossibility) to 1 (certainty).

A priori probabilities can be calculated from antecedent information and before the experiment or the event in question, such as probabilities of coin tosses.

Relative frequency probabilities, on the other hand, are derived from the empirical data of a sufficiently representative, random sample. Usually a reference class and attribute of interest are involved, and in theory the probability can be expressed as a numerical value, calculated as a fraction where the denominator is the number of members in the reference class and the numerator is the number of members of the reference class who have the attribute involved. Probabilities are assigned to events on the basis of available evidence or sample, and therefore may be different when people have different sized data.

But is also possible to view a priori probabilities as relative frequency probabilities: hence the probability of heads in a fair coin toss at 0.5 can be conceived as the relative frequency of that outcome in repeated experiments of coin tosses over many instances, and as the repeated experiments approach infinity supposedly the numerical value will approach 0.5.

Advocates of the frequentist interpretation of probability might contend that a priori probabilities do not exist, but are ultimately explained by relative frequencies. It is interesting that Ludwig von Mises referred to objective probabilities as “class probabilities,” under the influence of his brother Richard von Mises (1883–1953), a proponent of the frequentist interpretation of physical probabilities.

I assume (I could be wrong) that if Bayesian probability uses frequency probabilities, it may also be able to yield objective probabilities.

A fundamental point is that risk (as opposed to uncertainty) is associated with objective probabilities, either a priori probabilities or relative frequency probabilities, when a numerical value can be assigned, as Frank Knight argued (though Knight’s terminology was potentially misleading as he also called risk “measurable uncertainty”).

Post Keynesians would argue that risk is not the relevant concept in many entrepreneurial investment decisions, but uncertainty.

(2) Subjective Probability (or evidential probability/Bayesian probability)
In instances where probabilities of events cannot be analysed in terms of relative frequencies or because the events are unique and cannot be included in a reference class, probability theory has been developed that measures “degrees of belief,” and that can be termed “subjective probability.” The usual procedure for this is some form of Bayesian probability theory.

In neoclassical economics, subjective probability theory was developed from the work of John von Neumann, Oskar Morgenstern, Frank Ramsey, Bruno de Finetti, and Leonard J. Savage, the latter of whom (drawing on Bayesian probability theory as well) formulated a formal model of decision-making where optimal decisions are made to maximise expected utility, and probability distributions are given by subjective evaluations.

But even here uncertainty is seen as a state of the mind, not as a state of the world, and ultimately Walrasian general equilibrium theory in its various forms requires real, objective probabilities to actually exist for events in economic decision making, and for the subjective probabilities of agents to converge towards these objective probabilities over time.

Curiously, though being subjectivists, Austrian economists reject the expected-utility representation of decision making under uncertainty in neoclassical economics (Langlois 1994: 118). We should also note that Ludwig von Mises’s “case probability” is not really the same thing as Bayesian subjective probability. Case probability is a purely subjective form of probability and Mises argued that “case probability is not open to any kind of numerical evaluation” (Mises 1998: 113). By contrast, Bayesianism does give numerical values to evidential probabilities, even if these are deemed subjective, but are updated and revised in light of new evidence.

(3) Uncertainty
In understanding uncertainty, the distinction between ergodic and non-ergodic processes is important. For neoclassical theory, reliable knowledge of the future requires the assumption of the ergodic axiom. Ergodicity is a property of some process or phenomenon in which time and/or space averages or attributes of that system either coincide for an infinite series or converge as the finite number of observations increases (Dunn 2012: 434). Thus a sufficient sample of the past can be said to reveal the future in an ergodic process.

But, for Post Keynesians, the complications involved in assessing the ergodic or non-ergodic nature of an economic process might be considerable, especially as there exist:
(1) genuinely ergodic economic processes/phenomena;

(2) genuinely non-ergodic economic processes/phenomena;

(3) economic processes/phenomena that appear ergodic for short periods of calendar time, but may change. (Dunn 2012: 435).
For example, non-stationarity and Shackle’s “crucial decision” concept in decision making are sufficient conditions for non-ergodicity, but not necessary conditions (Dunn 2012: 435–436). Future events or processes that are created by human agency are, above all, candidates for non-ergodicity.

Events where objective probabilities exist imply an ergodic world or a justified use of the ergodic axiom. Information from past and present data series should allow a probability estimate that approaches the objective numerical value as the data increases, even for future events.

Keynesian uncertainty (in the sense of Keynes and Post Keynesianism) stresses the unknowable nature of the future and the inappropriateness or profound limitations of probability theory.

Although there is not an exact equivalence between all the various concepts below (and perhaps some important differences), these concepts of uncertainty are roughly similar to Keynesian uncertainty:
(1) Knightian (unmeasurable) uncertainty;

(2) Misesian case probability;

(3) G. L. S. Shackle’s radical uncertainty;

(4) Ludwig Lachmann’s radical uncertainty;

(5) Austrian “structural uncertainty” (Langlois 1994: 120);

(6) Loasby’s partial ignorance, and

(7) O’Driscoll and Rizzo’s genuine uncertainty.
When the idea of fundamental uncertainty is understood as a crucial one for economic science, the next question is: how do economic agents act and make decisions under uncertain conditions?

George L. S. Shackle developed a theory of decision making under uncertainty that dispensed with probability theories in describing such behaviour, and this was a project derived from the work of Frank Knight and Keynes. In contrast, as we have seen, mainstream neoclassical economics via Arrow adopted the use of subjective probability in decision making theory, and effectively denied the (1) risk versus (2) Knightian/Keynesian uncertainty distinction.

Neoclassical theory was influenced by the work of Frank Ramsay and Leonard J. Savage and essentially went down the path of subjective probability theory with a Bayesian flavour.

I conclude by posing some other questions that seem important to me:
(1) what is the contribution and value of Gilboa and Schmeidler’s non-additive probability approach to decision-making under uncertainty?

(2) to what extent did Ludwig von Mises follow the frequentist interpretation of probability of his brother Richard von Mises?

(3) Knight made a distinction between “statistical probability” and “estimated probability.” Is “estimated probability” more or less “subjective probability”?

(4) What is the significance of Daniel Kahneman and Amos Tversky’s critiques of standard economic decision making theory, and that of Daniel Ellsberg in Risk, Ambiguity and Decision (2001)?
Copi, Irving, Cohen, Carl and Kenneth McMahon. 2011. Introduction to Logic (14th edn.). Prentice Hall, Boston, Mass. and London.

Dunn, S. P. 2012. “Non-Ergodicity,” in J. E. King (ed.), The Elgar Companion to Post Keynesian Economics (2nd edn.), Edward Elgar, Cheltenham, UK and Northampton, MA. 434–439.

Langlois, R. 1994. “Risk and Uncertainty,” in Peter J. Boettke (ed.), The Elgar Companion to Austrian Economics. E. Elgar, Aldershot. 118–122.

Mises, L. 1998. Human Action: A Treatise on Economics. The Scholar's Edition. Mises Institute, Auburn, Ala.

Runde, Jochen. 2000. “Shackle on Probability,” in Stephen F. Frowen and Peter Earl (eds.), Economics as an Art of Thought: Essays in Memory of G. L. S. Shackle. Routledge, New York.

Skyrms, B. 2010. “Probability, Theories of,” in Jonathan Dancy, Ernest Sosa, and Matthias Steup (eds.), A Companion to Epistemology (2nd edn.). Wiley-Blackwell, Oxford. 622–626.


  1. Well, from what I have gathered, it is possible to further categorise advocates of Bayesianism into "Subjective Bayesians" and "Logical Bayesians". J.M. Keynes would fit into the latter.

    Also, with regard to G.L.S. Shackle...he argues that no probability theory can be used, full-stop. This would include J.M. Keynes's contribution, which itself is a seminal work in the so-called "Logical Theory of Probability".

    To answer your questions:

    (1) what is the contribution and value of Gilboa and Schmeidler’s non-additive probability approach to decision-making under uncertainty?

    Well, if my memory serves me correctly, their work is a theoretical response that allows for an explanation of the Allais Paradox and the Ellsberg Paradox and incorporates Subjective Expected Utility decision theory. In other words, on a theoretical level, their work is meant to be a more comprehensive decision theory.

    (2) to what extent did Ludwig von Mises follow the frequentist interpretation of probability of his brother Richard von Mises?

    I'm not sure, but from what I've gathered from a few disciples of the Austrian School, apparently Ludwig von Mises did follow his brother's conception of probability very closely.

    (3) Knight made a distinction between “statistical probability” and “estimated probability.” Is “estimated probability” more or less “subjective probability”?

    Well, I can't answer this question. What doesn't help is the fact that Frank H. Knight apparently chose not to cite any of the philosophical literature available to him during the time he was working on what would become Risk, Uncertainty and Profit. Also, I believe that in what papers he left behind that survives, he didn't leave behind much of a clue what thinkers' books or writings he read that influenced his views.

    (4) What is the significance of Daniel Kahneman and Amos Tversky’s critiques of standard economic decision making theory, and that of Daniel Ellsberg in Risk, Ambiguity and Decision (2001)?

    If my memory serves me correctly, Daniel Kahneman and Amos Tversky's research and Daniel Ellsberg's research share a similarity in that they highlight that the issue of framing is important when a situation is presented to a decision-maker. The Ellsberg Paradox also demonstrates that the axioms of choice as formulated by Subjective Expected Utility decision theory are easily violated in his experiment because S.E.U. decision theory's axioms of choice explicitly define probabilities to be linear and additive in properties. More precisely, the "Sure-Thing Principle" is violated in the Savage Axioms, thus leading to an anomaly that requires an explanation and fundamentally changing the axioms of choice.

    1. Thanks for this comment.

      Do you have any good references for the differences in types of "subjective/evidential probability", or what you call the division between "Subjective Bayesians" and "Logical Bayesians"?

    2. Unfortunately, I can't name any good secondary sources off the top of my head. However, I think there are writings by a few scholars that might provide some sort of answer: Irving John Good and Sir Harold Jeffreys.

  2. (3) "Knight made a distinction between “statistical probability” and “estimated probability.” Is “estimated probability” more or less “subjective probability”?"

    As far as I know there is overlap. But I think that Syll and Davidson are correct: for Knight, I think uncertainty means something slightly different than for Keynes.

    (4) "What is the significance of Daniel Kahneman and Amos Tversky’s critiques of standard economic decision making theory, and that of Daniel Ellsberg in Risk, Ambiguity and Decision (2001)?"

    I think it is fairly well-recognised that Kahneman and Tversky's experiments negate Expected Utility Theory. EUT thus lives on in a sort of undead fashion, still taught in classrooms but basically recognised to be false (or "flawed", as people who teach it like to say) -- in this regard I think that it is very similar to the Efficient Markets Hypothesis which was also disproved empirically by behaviorist types (specifically, Robert Shiller).

    In saying that, I've read some of Kahneman's book and actually turned down an opportunity to interview him. I think his psychology is pretty crude, to be honest. There's a lot of funny stuff in his book that all seem to be based on a fundamental confusion between logical and lived/historical time. I was debating whether to raise these points in the interview but I thought that it would be unfair to engage in critique while interviewing someone, so I turned down the interview.

    Do his experiments disprove EUT? Yes. But EUT should have been self-evidently false from the outset. Also, I think there are far more compelling critiques of EUT than Kahneman's experiments/thought experiments. Nevertheless, if you want to critique EUT Kahneman et al are your go to guys due to the fact that they got a Nobel... sorry, Swedish Bank Prize. ;-)

    1. While I would not have turned down the opportunity to interview Daniel Kahneman, I would say that there have been better critiques of S.E.U. decision theory that should have been recognised earlier. Had Daniel Ellsberg published his doctoral thesis within one to five years of when he had completed it in 1962, the development of the economics profession in the second half of the 20th century would have been different, and he might have received a Nobel Memorial Prize in Economic Sciences by now - and the research of Daniel Kahneman and Amos Tversky would not be as well-acclaimed as they are today.

      That stated...have you read any of Kahneman and Tversky's research originally published in scholarly outlets, Philip Pilkington? It's better to go back to the original source whenever possible, IMO.

  3. I'm sorry, but Bayesian probability is bullshit. Economists need to read something good in this regard, perhaps Mario Bunge's "Types of probability".

    Both the "Bayesian" and the frequentists interpretations of probability are idiotic and more importantly semantically impossible interpretations of probability (probability is just a subset of measure theory, and you can't force neither "reading" into it).

    1. Is Mario Bunge's "Types of probability" a paper? If so, where is it published and when?

    2. Or presumably you mean this?:

      Mario Bunge, 1988. “Two Faces and Three Masks of Probability,” in Evandro Agazzi (ed.), Probability in the Sciences. Kluwer Academic, Dordrecht and London. 27–50.

    3. Sorry, didn't notice you answer me before :).

      I mean this: "Four concepts of probability" (

      But he actually discusses this issue many times, for example, in his Treatise on Basic Philosophy (in particular, if I recall correctly, in volume III). There's another article called "What is chance?".

      I did not know about “Two Faces and Three Masks of Probability", he shows why "Bayesian Probability"doesn't belong in any science.

      He's also quite critical, as a side note, of neoclassical economics, calling them nothing but pseudo-science (in fact, he thinks neoclassical economics is the most dangerous of all pseudo-sciences).

    4. Thanks for the link to "Four concepts of probability" and other references. Will indeed read them when time permits!