tag:blogger.com,1999:blog-17232051.post7296643448387998057..comments2024-03-28T03:16:14.104-04:00Comments on Noahpinion: Priors and posteriorsNoah Smithhttp://www.blogger.com/profile/09093917601641588575noreply@blogger.comBlogger17125tag:blogger.com,1999:blog-17232051.post-3434035971834233392015-02-05T14:48:49.882-05:002015-02-05T14:48:49.882-05:00I am not Bayesian fan. I think Bayes wanted to co...I am not Bayesian fan. I think Bayes wanted to convert nonbelievers into believers and what a way to do that. One can make guess on P(T) (believe or not believe), but one can not refute the miracles P(E|T)! And, if one keeps believing in miracles, we can easily recruits folks to do crazy things, like chopping heads, burning alive etc. Once there, it does not matter what likelihoods are. KVnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-86850715440929915082015-01-31T18:09:40.429-05:002015-01-31T18:09:40.429-05:00Washing out the prior happens when the data are so...Washing out the prior happens when the data are so informative that the likelihood function converges to a sharp Gaussian. In this situation, frequentist and Bayesian answers basically always agree. It's the fact that this doesn't always happen that makes the whole question of which approach to use an interesting and important one.Anonymoushttps://www.blogger.com/profile/13594876772432068373noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-51100451961231623182015-01-31T11:31:40.873-05:002015-01-31T11:31:40.873-05:00Whenever Frequentists comment on Bayes, it tells y...Whenever Frequentists comment on Bayes, it tells you something about Frequentists and nothing about Bayes.<br /><br />Drop the idea that probabilities are frequencies. Real world frequencies are physical Facts, like the temperature of the sun's surface. They can be predicted, estimated and measured. <br /><br />Probabilities are tools for modeling uncertainty. If our Evidence doesn't determine some Fact exactly then there's some uncertainty which we model using P(Fact | Evidence). Our evidence may not tell us the exact temp of the sun's surface, but it gives us a range of reasonable values that are consistent with everything we do know about it. <br /><br />Prob distros model this uncertainty by placing their high probability region (probability mass) over the region of reasonable values. So if the sun's surface temp is reasonably between 5000k and 6000k, then you could use a N(5500, 500) to model that uncertainty. A Bayesian Credibility Interval formed from N(5500, 500) would contain the true temp. The smaller the interval, the less uncertainty naturally.<br /><br />The goal in modeling uncertainty is to find consequences which are true for almost every reasonable value of the true temp. If you use that N(5500, 500) to determine that some “hypothesis A” has very high probability, then that means A holds true for almost every potential temp in the range 5000k-6000k. Since the true temperature is in that range, then that's strong reason to think A is actually true.<br /><br />The best way to think of it is as a very intuitive and simple sensitivity analysis. Basically the high probability of A shows that hypothesis A is insensitive to the exact temperature as long as the it's in the high probability region of N(5000,500).<br /><br />So to select prob distributions take what you know and find a distribution that covers the right area. For example, if you're trying to create a prior used to search for a downed aircraft, you could use the max radius of travel based on how much fuel they had and place a uniform distribution over the entire disk. That way the true location is bound to be within that area. That defines your overall search region before considering anything else. <br /><br />Alternatively, you could ask an expert for their intuition and create a prior out of that. Of course, if the true location isn't in the high probability region of the expert's prior, then the prior is “wrong” and will lead to a failed search effort. <br /><br />Generally, creating prob distros is a competition between two competing goals. You want to the make the distribution as “informative” as possible about the true value, but if you shrink it's high probability area (region of plausible values) smaller than your evidence allows then you run the serious risk of missing the true value.<br /><br />Note, there is no difference between sampling distributions and priors. They both model uncertainty in the exact same way. When you use a NIID model for errors you're not saying that future errors will have a NIID frequency histogram. Statisticians almost never have that kind of knowledge and most of the time it's not true or even meaningful. <br /><br />What you're really doing is defining a region of plausible values for the one set of errors that actually exist in the data you actually took. You're saying the data you just measured had an error vector which resides somewhere in the high probability region of the NIID (a n-sphere with a radius of a few stdev'). As long as that's where those errors actually are everything is fine; future unrealized errors are irrelevant and can be absolutely anything. <br /><br />That's why NIID assumptions work better than they “should”. Errors of real measuring devices almost never have frequencies that look NIID over a long period of time. But they don't have to, because that's not what that assumption means and it's not what determines success! <br /><br />P.S. I respect Senn, but his description of Jaynes's views in that paper was so wrong he lost all credibility when it comes to identifying real Bayesians.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-66637847583633125552015-01-29T17:28:37.233-05:002015-01-29T17:28:37.233-05:00Let's make a deal. I'll do that if you sto...Let's make a deal. I'll do that if you stop constantly showing up here and at BV and posting "race and intelligence" crap! Deal?Noah Smithhttps://www.blogger.com/profile/09093917601641588575noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-981226122504014232015-01-29T10:29:51.214-05:002015-01-29T10:29:51.214-05:00Do a post about what happens to money when markets...Do a post about what happens to money when markets crash . does a conservation of wealth exist?grey enlightenmenthttp://greyenlightenment.comnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-3077186695645356682015-01-29T09:35:51.427-05:002015-01-29T09:35:51.427-05:00Every few weeks Noah goes back to Bayes! I though...Every few weeks Noah goes back to Bayes! I thought my links to Prof. Brown at Trinity University amplified that priors don't matter as long as...<br /><br />3.2 Washing Out of the Priors<br />The idea that P(T) could be based on a mere hunch may seem unsettling. After all,<br />different people may have very different hunches about the truth of a theory, and so may<br />begin the process with very different values for P(T)! In a way, however, this does not<br />matter very much. This is because of the phenomenon of the washing out of the priors.<br />If you and I begin with very different evaluations of T, but we agree on P(E|T) and<br />P(E|¬T), then our posterior probabilities will get closer and closer to each other the<br />more evidence we investigate. In the long run, we will end up with the same assessment<br />of T even if we started out with very different guesses.<br /><br />The operating constraint is:<br /><br />...If you and I begin with very different evaluations of T, but we agree on P(E|T) and<br />P(E|¬T),<br /><br />If Noah researches on the web, in the comment sections of his own blog, finds the article of Prof. Brown, and, sits down at one place and reads 9 page paper of Prof. Brown, he would not have these posterior farts. Better yet, just call up Prof. Brown and learn.KVnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-36594793280065748712015-01-29T07:56:04.186-05:002015-01-29T07:56:04.186-05:00"I don't think Senn's point really ha..."I don't think Senn's point really has any implications for study design"<br /><br />That is obviously not true, because his view is not compatible with the strong likelihood principle. Maybe you never accepted SLP in the the first place, but Bayesians of the sort to whom he refers mostly do.Phil Koopnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-53365231602565851392015-01-29T04:48:27.351-05:002015-01-29T04:48:27.351-05:00So... is Senn's blog post basically saying tha...So... is Senn's blog post basically saying that frequentist approaches are what we use when we can't agree on a Bayesian prior? This makes considerable sense to me, and it does indeed bridge the gap between Bayesian inference and frequentist hypothesis testing quite nicely.<br /><br />David: If you're looking at approaches to selecting priors, you might be interested in <a href="http://lesswrong.com/lw/dhg/an_intuitive_explanation_of_solomonoff_induction/" rel="nofollow">Solomonoff induction</a>. I'm pretty sure that the principle of maximum entropy, or something very similar, could be deduced as a special case of this.Corkscrewnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-15346906487440122172015-01-29T03:31:04.111-05:002015-01-29T03:31:04.111-05:00Who is the cutie in the "nested statistician/...Who is the cutie in the "nested statistician/ hall of mirrors" image? Is that Stephen Senn? I'm not certain, as the linked post had this rather different 2012 photograph: <br />https://errorstatistics.files.wordpress.com/2012/01/photo.jpgEllie Khttps://www.blogger.com/profile/11231840376889029260noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-31261296404529017782015-01-29T03:25:06.155-05:002015-01-29T03:25:06.155-05:00Oops, that should be "...why I AM wrong"...Oops, that should be "...why I AM wrong" (terribly embarrassing sentence in which to have a typo). <br /><br />Also, to be on topic, Deborah Mayo has an entire series of posts about that Normal Deviate here: <br />http://errorstatistics.com/?s=deconstructing+larry+wassermanEllie Khttps://www.blogger.com/profile/11231840376889029260noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-85926393547896093412015-01-29T02:14:14.338-05:002015-01-29T02:14:14.338-05:00Noah, have you come across the [maximum entropy pr...Noah, have you come across the [maximum entropy principle] (http://en.wikipedia.org/wiki/Principle_of_maximum_entropy) for choosing priors advocated by E.T. Jaynes? David R. Pughhttps://www.blogger.com/profile/09032073870730301659noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-40449333014831190082015-01-29T02:02:51.861-05:002015-01-29T02:02:51.861-05:00Excellent post! And some radical reconception of t...Excellent post! And some radical reconception of the whole field (relativity, quantum mechanics) should force a re-evaluation of priors, not a Bayesian updating! If you look at Bayes work itself, it is clear he is talking about situations in which we understand the experimental setup well enough to set priors!gcallahhttps://www.blogger.com/profile/10065877215969589482noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-67237417512238928212015-01-29T01:14:35.950-05:002015-01-29T01:14:35.950-05:00I don't understand what Gelman means by this, ...I don't understand what Gelman means by this, <br />"my work on the philosophy of statistics is intended to demonstrate how Bayesian inference can fit into a falsificationist philosophy that I am comfortable with on general grounds." <br /><br />The term "falsificationist philosophy" has an alarmingly glib connotation. Please tell me why I a wrong, for peace of mind?Ellie Khttps://www.blogger.com/profile/11231840376889029260noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-1016626610529676042015-01-28T21:51:25.216-05:002015-01-28T21:51:25.216-05:003. You could choose a bunch of different priors an...3. You could choose a bunch of different priors and see how sensitive the posterior is to the choice of prior. <br /><br />Bootstrapping?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-71292620166489244862015-01-28T19:01:39.707-05:002015-01-28T19:01:39.707-05:00My alternate answer is that the prior comes form t...My alternate answer is that the prior comes form the same place that the likelihood comes from.Stephen Gordonhttp://worthwhile.typepad.comnoreply@blogger.comtag:blogger.com,1999:blog-17232051.post-62118393315319037722015-01-28T18:28:08.603-05:002015-01-28T18:28:08.603-05:00Sure you can formulate *a* prior. Obviously you ha...Sure you can formulate *a* prior. Obviously you have to do so in order to carry out a Bayesian estimation. But what Senn and Gelman are saying is that you can't formulate *your own subjective prior*, like a Bayesian agent does. Noah Smithhttps://www.blogger.com/profile/09093917601641588575noreply@blogger.comtag:blogger.com,1999:blog-17232051.post-87160769099595974222015-01-28T18:17:21.416-05:002015-01-28T18:17:21.416-05:00If you can't formulate a prior, you don't ...If you can't formulate a prior, you don't understand the implications of your model, and have no business estimating it.Stephen Gordonhttp://worthwhile.typepad.comnoreply@blogger.com