The Quiz Daniel Kahneman Wants You to Fail
"In the December 2011 issue of Vanity Fair, Michael Lewis profiles Nobel Prize–winning psychologist Daniel Kahneman, who pioneered research into “heuristics,” or the shortcuts humans use when making decisions. At this link, take a 5-question quiz to see how your own mind works."
22 Comments:
Interesting reading, but I don't agree with one of his answers. If you have a group of 100 people and 50 of them are engineers and 50 of them are veterinarians, what is the likelihood that Sally (a woman) is an engineer?
Daniel Kahneman would aswer that it is precisely 50%. Most Carpe Diem readers would say the answer is 30% because women earn 30% of engineering degrees and 70% of vet degrees. For Daniel's answer to be correct, the group of engineers and veterinarians would need to be a non-representative sample.
>>> The knee-jerk reaction is to select answer C; we expect things to follow a proven pattern regardless of size. But size matters. A small sample size (i.e., the small hospital) will often contain extreme proportions, while a large sample size (i.e., the large hospital) will more likely reflect real-world distributions.
That's a ridiculous claim since they don't adequately differentiate between a "small" hospital and a "large" hospital.
How small is the "small" hospital? 50 patients a day? 100? 300?
I'd also suggest that the nature of the environment AROUND the hospital is likely much more critical -- is this small hospital in the middle of NYC or is it in Lesser Podunk, Wyoming?
Both of these factors make the argument about what the answer is to be rather statistically bogus.
>>> He shows no interest in political and social issues and spends most of his free time on his many hobbies, which include home carpentry, sailing, and mathematics.
Question 2 is equally ludicrous, as it suggests there's no difference between the typical interests of lawyers vs. engineers. If those descriptions were actually vetted against a real-world data set of lawyers and engineers for profession-neutral choices, I'd be rather surprised.
This "test" was constructed by a total retard.
A lawyer can raise the probability of winning in court on perceived biases (whether they're true or not).
"I'd also suggest that the nature of the environment AROUND the hospital is likely much more critical -- is this small hospital in the middle of NYC or is it in Lesser Podunk, Wyoming? "
Why? Is there something in Wyoming which results in a higher likelihood of bearing a son vs a daughter?
"Question 2 is equally ludicrous, as it suggests there's no difference between the typical interests of lawyers vs. engineers. If those descriptions were actually vetted against a real-world data set of lawyers and engineers for profession-neutral choices, I'd be rather surprised."
Yes, but then again this is a sample of 100 people. Probably not a significant sample size to describe "engineers" or "lawyers" with any confidence. So you can't know if these 100 are representatives of their population.
Someone gets an award for playing at word games?!?!
juandos: "Someone gets an award for playing at word games?!?!"
Why not? Is that any more amazing than spending one's entire career saying ba?
Well, there's 5 minutes I'm never getting back
I believe if you pick a card from and deck of cards the probability it is a heart is 25%. If I tell you the card you now hold is red, the probablility it is a heart is 50%. The question isn't the probability of originally picking a heart, but the probability of the red card you are holding being a heart. So unless engineers and lawyers are equally likely to have the listed qualifications, the probability the card belongs to an engineer is not 30%. I know it is easy to get probability problems wrong so I am open to correction.
Suppose you change the question to:
"A team of psychologists performed personality tests on 100 professionals, of which 30 were engineers and 70 were lawyers. Brief descriptions were written for each subject. The following is a sample of one of the resulting descriptions: I am an engineer."
The probability the professional is an engineer would be 100%. The question isn't what is the probability of picking one of the 30 engineering cards, the question is given the information on the card, what is the probability it is an engineer. Those questions are dealing with different sample spaces (to use the math term).
Well thanks for reminding me of that little CD posting Ron H...
Very good!
"Well, there's 5 minutes I'm never getting back"
Yeah, but it was necessary to enjoy another 5 reading the comments...
ranger275 is absolutely right.
yes, correct Bayesian analysis should take into account that there are more lawyers then engineers to begin with, but by no means the answer is 30% as if we never obtained any additional info
oh, now I read ranger's comment to the end :-(
no, if lawyer and engineer has the same probability to have the profile, the probability that it was engineer is exactly 30%.
in real life, though, let's say engineer has 50% probability to have this profile and lawyer has 20% probability, so an arbitrary
person from the group has 29% probability to have this profile, and correct probability that we got and engineer is
0.50*0.20/0.29 = 0.34
not 30%, but far below 50% even in these settings
Efimpp, we have no information beyond what is given in the question. We can't assume that an engineer is more likely to have those characteristics, particularly since we are dealing with such a small sample (on which we have no info on how it was selected).
So their answer is still correct.
Its not like cards, because in cards we know all the characteristics of that population.
The problem, AIG, is that the so-called "correct" answer on the lawyer/engineer question has its own presuppositions and biases: it presumes that none of the personality traits listed are relevant.
The actual correct answer is, "it depends upon the relevance of the items mentioned." Kahneman's answers would be correct if the question were "What is the probability that Jack is one of the 30 engineers, assuming that the above description is totally uncorrelated with a man being an engineer or a lawyer? This means that you can not use any other knowledge that you may have gained in your lifetime to answer the question."
Most people do -- and normally should -- bring a opinions to their answers: the real question is whether or not these opinions are correct, and, if so, what the level of significance is: the opinions should be open to correction.
Kahneman has some good insights and has done some good research; this is not a good example.
OBH: "That's a ridiculous claim since they don't adequately differentiate between a "small" hospital and a "large" hospital."
Yes they do: a "large" hospital will always be bigger than a "small" hospital.
"How small is the "small" hospital? 50 patients a day? 100? 300?"
It doesn't matter. It will be fewer than the "large" hospital. A larger sample size should more closely reflect a real world distribution.
"I'd also suggest that the nature of the environment AROUND the hospital is likely much more critical -- is this small hospital in the middle of NYC or is it in Lesser Podunk, Wyoming? "
Both hospitals are in "a town". We have no additional information, and no reason to suspect any other influence on the boy/girl ratio.
For answer B. to be correct, we ARE asked to assume that the larger hospital serves more maternity patients than the smaller one.
to AIG:
> Efimpp, we have no information beyond what is given in the question...
not enough to calculate - true.
but we can make assumptions - like these were more or less randomly selected lawyers and engineers (do you agree in another problem that "big hospital" has a bigger maternity ward then a "small hospital")?. And that yes, the profile is more suitable for engineer then for a lawyer (do you agree with it?)
so, the answer still is: "definetly more then 30%"
CBI: "The problem, AIG, is that the so-called "correct" answer on the lawyer/engineer question has its own presuppositions and biases: it presumes that none of the personality traits listed are relevant."
None of the traits, taken individually, is a good indicator of a person's profession. Why would they be if taken as a group?
"The actual correct answer is, "it depends upon the relevance of the items mentioned." Kahneman's answers would be correct if the question were "What is the probability that Jack is one of the 30 engineers, assuming that the above description is totally uncorrelated with a man being an engineer or a lawyer? This means that you can not use any other knowledge that you may have gained in your lifetime to answer the question."
To be told that the description is not helpful, would defeat the entire purpose of the quiz.
"Most people do -- and normally should -- bring a opinions to their answers: the real question is whether or not these opinions are correct, and, if so, what the level of significance is: the opinions should be open to correction. "
The whole point of the quiz is to show that we DO bring our own additional information to bear on the question and that sometimes it leads us to incorrect conclusions. This is made clear in the explanatory note.
to efimpp,
I'm not sure if we disagree or not. My point was the information on the card changes the probablility which I think we agree on. In my modified senerio where the card said "I am an engineer" the probability the card belonged to an engineer would have to be 100% unless we assume some engineers and/or lawyers lied on the card. The probability you get the card in the first place is not 100%, but once you have the card the probability it belongs to an engineer would be 100% which I think is what the original question was asking.
I do disagree with your 34%. If 50% of the 30 engineers and 20% of 70 lawyers have this profile then 15 engineers and 14 lawyers have this profile. So if you have the card in your hand it must be one of these 29 cards and the probability it would be one of the engineering cards would be 15/29 which is 51%.
to ranger275
for the record, you are right in your last comment, and I just put a wrong number in the formula, it should be
0.50*0.30/0.29 = 0.51
Question 2 is wrong (I assume that the quiz writer changed K-T's example to make it more interesting and in doing that inadvertently added informative data to the profile -- an error that K-T took pains to avoid in their problems). Explained here:
http://sitacuisses.blogspot.com/2011/11/how-to-misunderstand-technical-material.html
Cheers,
J
Post a Comment
<< Home