Sean Carroll is one of my favorite science bloggers, and you should definitely check out his blog, Preposterous Universe, if you have not done so already. But I don't agree with his idea that it's time to toss out the notion of "falsifiability" in science.
Carroll does make some good points. For example, some theories might be falsifiable in principle but not in practice, given technological limitations. For example, take the tiny strings in string theory, which we could only see if we had a particle accelerator the size of the galaxy. The fact that we'll never build a machine that big doesn't mean the strings aren't there.
Another point Carroll makes is that sometimes, things that aren't currently falsifiable eventually become so. If we toss out ideas too soon because we haven't yet figured out a way to test them, we may cut ourselves off from understanding things.
But Carroll is a bit unclear as to what he means by "falsifiability". For example, he writes:
The cosmological multiverse and the many-worlds interpretation of quantum mechanics posit other realms that are impossible for us to access directly. Some scientists, leaning on Popper, have suggested that these theories are non-scientific because they are not falsifiable.
The truth is the opposite. Whether or not we can observe them directly, the entities involved in these theories are either real or they are not.What does he mean by "real"? What does it mean for something to be "real", but to be so hidden from our Universe that we'll never be affected by its "reality"?
Anyway, it seems to me that there are at least three different kinds of "non-falsifiable" theories:
Type 1: Theories that are so vague that they can be used to "explain" anything (e.g. "Every event in history is the result of inevitable historical forces.")
Type 2: Theories that are completely untestable in principle (e.g. "There's a particle that can't possibly interact with anything in our Universe.")
Type 3: Theories that are currently untestable given the limits of our technology.
I propose that we treat these three types differently. As for Type 1 Unfalsifiable Theories, we should run around with squirtguns, soaking anyone who wastes our time with these. The Type 2 theories, we know we'll never need, so scientists can safely forget about them, while amateur philosphers endlessly debate the proper meaning of the word "existence".
But what about the Type 3 theories? We might be able to use them someday, but not yet. Do we throw them out (thus risking cutting ourselves off from the truth), or do we use them as long as they're plausible and cool-sounding (thus risking using bad theories)?
I say we do neither. We humans are trained to use two-valued logic, which labels properties either "true" or "false". But I believe science needs a third category: call it "shrug". True, false, or shrug. The things in the "shrug" category should be placed in a mental bin for safekeeping until such time as they might prove useful to humankind, but not used until there exists enough evidence to verify their usefulness to our own satisfaction.
In fact, I think that three-valued logic is a crucial and innovative feature of scientific thought. You need to have that intermediate category where you defer judgment. Call it "shrug-ability". (Note that sometimes the world forces us to make discrete policy decisions, in which case you can't just shrug. But I'm talking about the decision to use a scientific theory, which is typically less urgent.)
Of course, there's also the question of whether our conclusions about scientific theories should be discrete at all. In reality, of course, they aren't. "Accepting" and "rejecting" theories are just mental shorthand. "Shrugging" at theories is also mental shorthand, it just reminds us to consider our confidence levels in addition to our point estimates.
Anyway, Scott Aaronson has some other good thoughts on falsifiability.
One can't really say there are any Type 2 theories without realizing they can't be turned into Type 3 theories with little effort, gravity leakage between multiverses or portals between them for example, so while it may not be worthwhile spending much time on them, one may want to consider them as stepping stones to more fruitful or useful ones.
ReplyDeleteType 2 theories are Type 3 theories that choose to voluntarily disqualify themselves from empirical tests!
DeleteNo, just ones that haven't risen to the challenge of figuring out how they might be yet.
DeleteOf course, rejecting ideas solely because they are not falsifiable is absurd--there exist, after all, many worthwhile disciplines which are not sciences.
ReplyDeleteLike KUNG FU
DeleteYes, and Popper wrote himself in a frenzy on several occasions that falsifiability pertains to a demarcation criterion between science and non-science. His idea of science was heavily influenced by physics, so some social scientists who might be excluded by this criterion might feel that to be unfair. On the other side, I'll borrow a dictum by Popper himself: never argue about words. The important thing here is, IMO, not a skirmish about the meaning of the word "science", but to note that Popper suggested a demarcation criterion which - if accepted - has a real meaning, whater you call it it demarcates. On the other side, it is NOT a criterion of value (or what is called a "Sinnekriterium" in German): Popper does not (or not automatically) judge whatever is 'unscientific'. Though he clearly strived to develop a philosophy centered on topics that can be discussed in a quasi-falsifiable way, he did no make direct connections between what is 'scientific' and what is 'worthwhile.'
Deletebut scientists are terrified of creationists
DeleteSorry, meo fio; but sicentists are not "terrified of creationists." Mostly they feel sorry for them pathetically floundering and making their silly, half-baked arguments.
DeleteBarkley Rosser
I've always understood Popper as saying that science is concerned with contingent truths rather than necessary truths. That is to say, a number of deductions from a set of tautologies may be of interest to a logician, but it does not constitute an empirical science. His criticism of Alder, Freud, and Marx went along these lines. Their theories would hold no matter what observation we made, because they were so designed that they accounted for all possible experience. But a genuine scientific theory, in the Popperian sense, is one which accounts for all those facts of the actual world even though it does not cover all possible worlds.
ReplyDeleteIn relation to your previous post, this is also why I have a big beef with the methodology of Austrian economics. Insofar as the Action Axiom is a necessary truth (a tautology) it cannot be empirically useful. So it would be unfit for the founding principal for scientific study.
This comment has been removed by the author.
DeleteBtw. not all Carroll says is news: one of the most famous example thrown against Popper (by critics like Feyerabend) was the postulation of the (anti)neutrino by Pauli. Why, Mister Popper, Pauli clearly should have abandoned conservation of energy, momentum, and spin on empirical grounds- rather than to develop some crazy ad hoc-idea about a particle that magically has all the properties to save those laws and that he cannot observe? Well, that would have been unfortunate.
ReplyDeleteI know Popper has written about this, because I read it. Unfortunately, I don't remember his argument, ha! Anyway, the problem is that Carroll obviously didn't go very deep into the matter, either - and he is the one asking to abandon the criterion, so one would think he actually read what he talks about. If so, he would have found this episode that sounds, if I am not mistaken, an awful lot like an antecedend of a shrug-ability problem. I am very much convinced, Popper - knowing the difference between a logical demarcation criterion and real-life sciences - would have approved of it.
It's telling that Carroll dragged out String Theory, which has zero empirical evidence to support it and no known way to falsify it it empirically short of some ludicrously high-energy particle accelerators. It's not even science, just pretty mathematics that fits what we know of physics provided you also take on faith the belief that there are god knows how many extra dimensions involved.
ReplyDeleteScott Aaronson wrote:
DeleteI also think Sean might have made a tactical error in choosing string theory and the multiverse as his examples for why falsifiability needs to be retired. For it seems overwhelmingly likely to me that the following two propositions are both true:
1. Falsifiability is too crude of a concept to describe how science works.
2. In the specific cases of string theory and the multiverse, a dearth of novel falsifiable predictions really is a big problem.
Brett,
DeleteWhile a definite test that we can do with current tech has been proposed, there have been discussions by serious physicists of possible tests that might be done that would involve much less tech power than a galaxy sized accelerator, and less so for multiverses. Both the string theory and multiverse theories are ones that may yet become falsifiable with further thought, experimentation, and tech advance.
Barkley Rosser
They can become falsifiable? Given all the word games of what String theory is and is not and its proven ability to predict every single thing out there, nobody should take this stuff seriously. Especially after 30 years of playing with interesting, but so far empty symmetries. It's a disgrace what theoretical physics have become.
DeleteKryz,
DeleteThree are a whole bunch of people at the Perimeter Institute in Waterloo, ON working on alternatives to string theory, with the most famous one there, its director, being Lee Slonim. I think that they are all hoping that some day we shall figure out ways to prove or disprove their various compeing theories, although at this point this cannot be done. But simply saying that string theory is not disprovable in principle I think is quite wrong.
B,
DeleteYes, people do work on alternatives and that's obviously very hopeful. However, as the unhappy story of string theory shows, a failed paradigm can entrench itself so much that it will even try to redefine the meaning of failure and success and change the rules of the game.
Late-20th/early-21st century physics is in the process of jumping the shark. The "lets toss out falsifiability" argument should come with this image: http://fc06.deviantart.net/fs70/f/2011/165/f/3/jumping_the_shark_by_roseswebofnightmares-d3ixpfw.jpg
ReplyDeleteHeh.
DeleteI can't say you're wrong, hoss. I can't say you're wrong.
The word you are looking for is 'Scotch verdict'. Is this a forgotten term among today's youth?
ReplyDeleteI've never heard it.
DeleteGuess the reference is to the "not proven" verdict in Scots law, as opposed to "guilty" or "not guilty".
DeleteOr the bastard verdict, as it's affectionately known.
Not sure though, been on the scotch.
Should I be nasty and add the Duhem-Quine thesis to this discussion ?
ReplyDeleteOr should I be even more nasty and point out that Popper's theory is a type 2 theory ?
A problem with requiring falsifiability is that often we don't know whether something is falsifiable after studying and understanding it. If we would never study something in a scientific context before we know it's falsifiable, I think science could never have discovered anything. How long that takes, depends on the subject. Moreover, there is the problem of falsifiability in principle and in practice. String theory is not falsifiable in practice according to what we now understand about it. But it does seem to be falsifiable in principle, and we may find more practical ways to falsify it in the future when we understand more about it. So shouldn't it therefore not be part of scientific study? What is the time limit to wait for sufficient insight?
ReplyDeleteAs for multiverses, how about the opposite theory: the universe theory (i.e. the there that there is one universe.) As we currently understand it, quantum field theory predicts a cosmological constant that is many orders of magnitude larger than what we actually measure. This reasoning crucially depends on the assumption that there is one universe, because if there are many universes, the anthropic principle says that it can be very small, and cannot be very big. One could say that this falsifies the universe theory, and based on actual measurements (of the cosmological constant), even if we cannot directly see the other universes. So in fact, what is measurement? The problem is that the interpretation of any measurements we do are based on our theories and the understanding of them, and they often are very indirect (even our brains, which we don't understand very well, are involved.) So how indirect should a measurement be to fail to call it a direct observation?
Can we integrate this 3 part falsifiability division with the important philosophical designation, "true, but not interesting"?
ReplyDeleteI'd say that simply "not interesting" is sufficient. "Truth" is an ambiguous, loaded word.
DeleteQuantum Logic: propositions that are both true and false at the same time!
ReplyDeleteI have to side closer to Sean (incidentally an old teacher of mine from back in his old U of C days... doubt he remembers me though) on this one. My view, slightly different than either of you, is that the "demarcation problem" is not one of models; although to get my point across I might need to take a detour to a proper theory of knowledge... sorry, just bear with me a moment (I'll be quick as I can).
ReplyDeleteI think there is only one thing that it is possible for me to know; to use Descartes's phrase "Cogito ergo sum". Strictly speaking I can be a brain in a jar being fed sensory information or a day-dreaming supercomputer. But then, if that were the case, why would I imagine the world I do? That I am a physical being pecking at a keyboard is a model, but more than that, given everything what I know (what is in my head... sensory information and whatnot), that I am a thinking being surrounded by physical objects is the SIMPLEST explanation. Which is to say that everything that I call knowledge is more strictly understood as a model on which I've applied Okkam's razor (for those not familiar with it, that means, roughly "all else equal, the simplest explanation is usually the correct one). I'm not a philosopher, but I think this view is a version of pragmatism (as opposed to realism or empiricism) and you can read more about it than you ever wanted to know at the Magic, Myths and Money blog.
At any rate, what's the relevance? Well, to some extent, from this view, "science" is just the application of Okkam's razor "all the way up". Science is a method, nothing more, and there is no sharp demarcation between knowledge which is scientific and non-scientific. Not to put too fine a point on it, but the principle of consensus (unfortunately ignored by too many philosopher's of science but it's just a corollary of Okkam's razor) i.e. that scientists agree on one theory (since that is itself a data point which need be explained), well the principle of consensus seems a much better candidate for demarcation if you really want to insist there be one.
On falsifiability, your points (1) and (2) both run afoul of Okkam's razor as well. For example, any explanation relying on God is an over-determined model (since you need to explain God's motives, but then those so-called motives are doing all the work... so God can be used to explain anything). Those are the case of (1). Conversely, a model which contains inaccesible pieces is under-determined, like your ghost-particle example: anything could be happening in that "sector" and I couldn't know it. So there you go; what you and Popper call "falsifiability" as if it were a deep principle is better viewed as an attempt to match knowledge to explanations optimally (the simplest explanation is neither over- nor under-determined).
So there you go... and the only thing you need to give up is that notion that "truth" and "extant reality" are things that you necessarily have access to...
Showing that the statement "God exists" is not a scientific one was a major motive for many of the logical positivist philosophers, and while Popper spent more time arguing for the unscientific nature of Marxism, he also weighed in on the God question regarding falsifiability.
DeleteA problem here not mentioned so far is that there may be disagreement among scientists or intelligent observers regarding what constitutes a proper test of falsifiability. Thus, an old friend of mine once stood up in a room with many people and announced that he was testing the existence of God, and that if God did not make him die in some manner in the next five minutes that would constitute a falsification of the existence of God. He did not die in the next five minutes (indeed, remains alive), but most observers would dispute that this was a reasonable or decisive falsifiability test.
Barkley Rosser
I'm familiar enough with the philosophy of science to know that in broad terms I'm well within standard views... but not familiar enough to be confident in using the correct terminology or to correctly attribute views to others. I think I'm much more aggresive with the use of Okkam's razor than most, but then the point is only that all knowledge beyond (my own existence) shares this common feature that it is the simplest explanation to some more basic set of knowledge: a process that continues iteratively until that primitive level of knowledge that I am a thinking being (which I know by virtue of the fact that I think). Thanks to Tim Johnson, I'm fairly certain that this is the pragmatist school of thought, whereas I understand most philosophers of science are realists or empiricists.
DeleteSo, in response to your point, your old friend is (implicitly) using a model in which God will Smite those who deny Him to prove His existence (or something of that variety). The problem is that there is a spectrum of models for God (including, but not limited to nonexistence) which would also be compatible with the knowledge that your friend was not in fact Smited (I presume). This is the problem with any model that includes God: it needs to say something about Him to be a basis for knowledge (What are His motives for Smiting?). Now if your friend HAD been Smited that could be viewed as evidence of His existence, since an explanation that did not include God's existence would likely be more complicated. At that point, we could, presumably, test the hypothesis by repeatedly taunting God to determine whether a Smiting always resulted. At that point, the Vengeful God model would probably pass the principle of consensus, although I imagine there would always be some who argue that other explanations are simpler (certain utterances cause human beings to break out in severe electrical burns?).
This example is one of what I would argue are many ways in which the falsifiability heuristic can lead to mistakes.
In this classic post by Sabine Hossenfelder, http://backreaction.blogspot.com/2013/09/the-multiverse-is-not-paradigm-and-its.html, she distinguishes between two categories of multiverse. In the first category are multiverses that can be reduced (in principle) to a single universe by adding conditions determined by real-world observations. She puts string theories in this category. The second category is reserved for theories in which this is not possible; the "many-worlds" interpretation is an example, in her view. However, she qualifies this: "It might also turn out that these multiverses aren’t unobservable after all, so these ideas certainly merit some investigation."
ReplyDeleteWhich means there really isn't any point to the proposed third category of truth value: all values of scientific truth are "shruggy" anyway. Either on a continuum of truth, if one subscribes to your Bayesian philosophy, or else on a continuum of testedness of binary truth if one subscribes, say, to Mayo's Frequentist interpretation.
" all values of scientific truth are "shruggy" anyway"...
DeleteI suppose this is a simpler and more practical way of saying what I was trying to get at above. For example, one thing I forgot to say is that if the principle of consensus is your demarcation and if you are a bayesian (hmmm... do I need to be a bayesian if I'm a pragmatist?) then there is a spectrum of scientific "truth" (hate that word in this kind of context) since the degree of agreement within the science community can vary. At the same time, too much agreement can be viewed as evidence of an information cascade; so there is even in my formulation perhaps some special level of agreement.
Of course, as above, I'd reiterate that my preferred view is that there is no need for a demarcation principle to separate scientific from non-scientific knowledge, science is just a process for adding to knowledge. All knowledge is built in the same way: as economizing over models (or, if you'd prefer to see how this principle fails: no one has knowledge of God).
So, a bit more, having already piled on with several. I would note that my late father coauthored a book over a half century ago with Atwell Turquette on "Multi-Valued Logics." This is an old topic, and there may be more alternatives to "true" and "fals" than "shrug," many more. Indeed, one alternative is to have a continuum of possibilities identified by probabilities ranging from zero to one, although one can get into an meta-argument here about whether these probabilistic statements themselves are true or false... :-).
ReplyDeleteAnother point in response to J. Edgar is that a view that says a statement can be both true and false, putting aside this probabilistic way of doing so, is not called quantum logic (nice try at neologism), but intuitiionist logic, and it has been around for over a century now. Its roots lay in mathematic debates over Cantor's set theory and constructivism from th e19th century, but the most famous codifier and advocate of intuitionism was none other than L.E.J. Brower, the same responsible for the famous fixed point theorem von Neumann first applied to prove existence of equilibria in game theoretic and economic models.
BTW, I have a paper on some of these latter issues, "On the Foundations of Mathematical Economics," on my website and published in New Mathematics and Natural Computation in 2012.
J. Barkley Rosser, Jr.
"Brouwer" not "Brower" in last comment. BTW, that is Dutch for "brewer."
ReplyDeleteJBR
Forgive me if I'm wrong but wasn't the whole issue of deciding which theories to throw out and which to keep settled with Lakatos' concept of the 'research programme'?
ReplyDeleteI normally love your blog Noah but when you start talking about the philosophy of science you seem to be re-inventing the wheel with a lot of this stuff.
I have ovecommented on this thread, but one more. Posing "shrug" as the third option really does a disservice to the most likely alternative options to basic "true" vs "false." What it suggests is an attitude, that you do not care whether it is true or false. Now, maybe that is what you meant to imply, Noah, but it is not philosophically serious or interesting. It may be reasonable about some of the cosmological sorts of examples you have introduced such as "Is string theory true?" or "Are there other universes?" which for most people the truth or lack thereof will never affect them by one iota.
ReplyDeleteBut the more serious matters are ones where people do care about the answeres, the answers are important, but for one reason or another presented in various comments above, the standard Aristotelian dichotomy is not reliable.
JBR
You forget a 4 type of falsifiability. Some things might become falsifiable if you think carefully about a theory's implications. The perfect example is Bell's inequality.
ReplyDeleteInteresting post. I don’t think that the key test of understanding most ideologues lies in understanding their ideas. The ideas themselves are often quite simple. Rather it’s in understanding the exaggeration around the ideas.
ReplyDeleteFor example, most people would agree that governments sometimes make mistakes. However, certain ideologues believe that governments always make mistakes. Most people would agree that capitalists sometimes exploit their workers. However, certain ideologues believe that capitalists always exploit their workers. Most people would agree that managers sometimes add little or no value to their companies. However, certain ideologues believe that managers always add little or no value to their companies.
After the exaggeration arising from replacing ‘sometimes’ with ‘always’, ideologues then feel the need to start verbal wars with non-believers, even people who are sympathetic to the original ‘sometimes’ ideas. In these verbal wars, when faced with real-world counter-examples to their ideology, the ideologues are forced into a position where they have to argue that other ideas are always false. Again, most people would agree that these new ideas are sometimes false but disagree with the exaggeration.
Economic ideologues are akin to Trekkies. Many people like watching Star Trek on TV. However, Trekkies take that to extremes by dressing up as aliens; talking to each other in the Klingon language; starting verbal wars with anyone who like Star Wars; and boring everyone else by talking about Star Trek all the time. Even if a normal person likes Star Trek, the Trekkies will always find them out when it turns out that they don’t know an obscure detail that would be known only to a fanatic.
The main difference between economic ideologues and Trekkies is that economic ideologues see the world entirely in terms of competing ideologies. If you aren’t 100% for their ideology then you must be 100% against it. Economic ideologues see everything in black and white. There are no shades of grey. There is no ’shrug’ in economic ideology.
Sorry. I've posted this on the wrong thread. Should have been on the Turing machine thread. I came over to this thread to remind me of 'shrug' and forgot to go back
DeleteYup. To paraphrase Carl Sagan, it's ok to wait until the evidence is in. That is, to shrug.
ReplyDelete