Peer Review. Game On.
October 26, 2009In recent weeks, some commenters on this and other blogs have tried to argue that the peer review system in climate science is broken; that under normal circumstances, they might trust peer review, but for some reason (given the technical narrowness of peer review in dendrochronology, for instance) peer review couldn’t be trusted.
Today, we have news through Roger’s blog that at least some critical responses have been attended to through the proper peer review channels.
Among other things that this points to is the non-brokenness of peer review. So, I reiterate: if the failings of a given study are so grave as to undermine the study, then those failings should, and apparently can feasibly, travel through the appropriate channels of peer review.
Cruel Mistress 
Please don’t misunderstand my position on academic journals. I’ve never suggested that blogs supercede academic journals.
However, it’s a stretch to claim that this particular incident is an achievement of peer reviewers appointed by journals. Quite the opposite. In Sep 2008, I observed at Climate Audit that Mann et al 2008 had used several important series (Tiljander) upside down. Ross McKitrick and I submitted a comment to PNAS making this observation. Mann denied that he had used them upside down and PNAS took no steps to resolve this elementary issue.
Kaufman also used the same data upside down. None of the peer reviewers at Science noticed or objected. I posted Kaufman’s upside down use of the Tiljander at my blog and notified Kaufman in early Sept, also asking Kaufman for certain series of original data. It now seems that there were errors in 4 of 23 series, not picked up by the appointed peer reviewers. Kaufman refused my request and told me not to contact him further and appears to have taken no further steps at the time.
My blog posts attracted the attention of a Finnish paleolimnologist, Atte Korhola, who criticized the Kaufman article in much stronger language than I had used on a Finnish language blog. His comments were translated into English by a Finnish reader of CA and posted at CA on Oct 2. Korhola also praised Climate Audit, noting that careful work had been there, and criticized the juvenile response to these issues at realclimate. Kaufman’s draft corrigendum is dated Oct 7, several days later.
Thus far, no correction has been issued by Mann et al.
I support the work done at academic journals and the existence of a well-indexed system of publications. In paleoclimate, peer review at the journal level is relatively cursory and was inadequate to identify the upside down use of this series in Mann et al 2008 and later in Kaufman et al 2009. I support the regular and prompt archiving of data and detailed methodology at the time of submitting an article for review, a practice already established in econometrics.
I didn’t actually have your work in mind. It’s clear to me that you’re doing a lot to try to make your points and your criticism as clear as possible. I think you should be applauded for that. I was responding to some other commenters who have less faith in peer review. My view is basically that peer review is the best we’ve got at this point, and that we can’t simply rely on blog coverage to evaluate the complex claims that circulate.
Ben,
What other work are you talking about? McIntyre’s word was the subject of Roger’s post.
Not so much other “work,” so much as comments. There were a few comments in some of the other threads that suggested that peer review was broken. That’s what I had in mind. I don’t really want to go back and dig them out, but they’re in the past hockey-stick threads.
Just to clarify that the matter under discussion here doesn’t even come close to being a “failing (..) so grave as to undermine the study”. Kaufman et al note precisely the opposite in their corrigendum. McIntyre’s clear expression of disappointment about this reveals his political agenda, and the same can be said about RP Jr.’s continued ambulance-chasing.
How was McIntyre being clearly disappointed?
Simple enough: He wants more and isn’t getting it, although to be fair you have to dig down under the sarcasm to find the disappointment.
Korhola is an interesting case. As best I can tell, he’s boxed himself into a minority contrarian position in the recent-past paleoclimate community by defending his own peat bog-based dating research that finds late-Holocene local temperatures in Finland at odds with other work. Most everyone else seems to think his results are crap. Of late he took the interesting step of co-authoring with his politician wife (a center-right MEP, noting that in Europe center-right means something rather different from what it does in the U.S.) an advocacy article on climate science. In the U.S. such a move would be perceived as blindingly inept, and I’d be interested to know if the same is true for Finland/Europe.
Ben –
Notice the modus operandi of propadandist Steve Bloom. “Most everyone else” thinks his [Korhola’s]results are “crap”. “Most everyone else” not identified or enumerated. And the results are not wrong, or mistaken, but “crap”. He has “boxed himself into a corner” by finding results that are “at odds with other work”. Why are the others not in the box in the corner? Maybe their work is “crap”. To complete the drive-by smear he notes that his wife is a “center-right politician” (cue the gasps of horror). Moreover, authoring a climate science advocacy article (whatever that is) is “blindingly inept”. Maybe he should have got arrested protesting coal, like Hanson.
There, in capsule form, is alarmist propaganda.
Ben –
Roger’s take on this is not correct. Peer-review failed to uncover the error. The author learned of it on Climate Audit, made the correction, but failed to cite the source. How is peer-review vindicated?
Don’t know. Just going off of what Roger said. You seem to have different information. Since the facts are now contested, I’ll lay off the facts and instead try the conditional: “If this is going to appear in the peer reviewed literature, then something about the peer review process is working.” How’s that?
It’s great that this will end up in the peer-reviewed literature. The problem is that it got there by a route that was external to the peer-reviewed literature.
BTW, it’s not right to expect that this sort of error should be caught in peer review. The scandal is that after M&M alerted Science to the problem, Mann was allowed to escape with a ridiculous assertion that “the sign of predictors doesn’t matter” and that the issue was “bizarre”. Anyone with a smattering of knowledge of math and science could see through this claim.
Moreover, Kaufman went ahead and did the same thing in his initial submission. IMO, it was only Korhola’s comment that forced Kaufman’s hand.
I bet you anything that Steve Bloom cannot provide any evidence to support this comment about Atte Korhola:
“Most everyone else seems to think his results are crap.”
Put up or shut up time Bloom.
Sorry, the anonymous are entitled to no service in that regard, the idea being that if I’m going to go to the trouble of refuting your fact-free snark I want to be rewarded with a documented reduction in your credibility.
Oh, aren’t you cute. I’ve got a novel idea. Maybe your readers are entitled to the documentation to back up your claim. Maybe the proprieter of the blog is similarly entitled. Maybe the dictates of your conscience demand it.
zing truth
Steve Bloom: I am not anonymous. Please provide me with the references.
Never heard of you, which makes you effectively anonymous.
And your reply to Tom C?
Steve: I have not heard of you, either. Plus, you have hinted at references you will not provide.
That makes you a pompous git.
zing truth. twice in one thread. keep it up, Bloom.
Steve Bloom, I am not anonymous, and you’ve attempted to smear one of my colleagues. So I too would value your back up information on your allegations.
It is unseemly at best for a Sierra Club official to go around make unsubstantiated allegations on a blog about an internationally respected scientist. It will give the Sierra Club a bad name, and unfairly represent the good people of the SC.
So how about providing some references or just admitting that you are making stuff up?
Imposter!
Tell you what, Roger, you give me some indication that you’ve made an effort to keep up to date with the broad recent-past paleo literature, conference proceedings etc. (how I got the impression I have about Korhola, BTW) as contrasted to just the material McIntyre has pointed you toward (as with the assorted flying monkeys above) and I’ll fix you right up.
Re your various pathetic aspersions, nice try.
Its kinda odd, that one side of the debate (Romm and Bloom) insists on being able to humiliate the opposition, and that the opposition should be of a high profile, before doing so.
If you had facts to back your position Steve, you would win more adherents.
Saying you have something, but not presenting it (with 3, count ’em 3, different excuses), makes you look weak and ineffectual. And not a little disingenuous.
I sincerely hope your fund raising skills are better than this, Steve, as the Sierra Club does fill an important function in society.
Bluff called.
Ooh, Roger’s got me over a barrel. 🙂
This is fun to watch. As someone recently said about someone else not long ago, Korhola’s crawled far, far out on a limb. The marginal science aside, the bit where he trashes Hansen is sure to make Roger nostalgic.
Steve Bloom: That link to Korhola does not support your case at all.
Do you disagree with Korhola’s following statements (some exact quotes, some paraphrased?
– Warming has increased in the arctic
– Permafrost is melting
– glaciers are in retreat
– sea levels are rising
– the means to mitigate warming must be updated continuously
– There is a need for public support against warming. (my bold)
– We need people like Jim Hansen (exact quote, my bold)
Or perhaps you disagree with some of the sources which he does not dispute, but uses as references:
– Rahmstorf 2007
– Mann et al 2008
– Briffa et al 1992
Perhaps you disagree with his methodology in using bogs as a proxy.
– he details his methods
– he describes the analysis done, using accepted practices in the literature (Barber et al 1994, Speranza et al 2000, Mauquoy et al 2002)
– he compared his proxy to other, published and accepted proxies.
Lastly. Do you disagree with his statement that warming in the 20th century is different than past warming events, and is probably due to an anthropogenic component?
Based on this reference, Roger does indeed have you over a barrel. The barrel is also filled with cement, with your feet firmly in the barrel, and sinking rapidly to the mid-Atlantic ridge.
Les, try going away and learning something about the field before running your mouth. But let’s get specific:
Korhola’s ideas are way outside of the mainstream, although he’s by no means a denialist. To name two major points, he finds a large-amplitude northern European MWP and a major solar role. Colleagues that agree with either of those are thin on the ground.
Re Hansen, what was it Korhola said is needed in addition? Listen carefully for that one. Also note the lack of a response to Korhola’s big laugh line.
Steve: your Les, try going away and learning something about the field before running your mouth. But let’s get specific:
I am anonymous, remember? How can you begin to question my qualifications?
Korhola’s ideas are way outside of the mainstream, although he’s by no means a denialist. To name two major points, he finds a large-amplitude northern European MWP and a major solar role. Colleagues that agree with either of those are thin on the ground.
MWP? He compared his proxy against 4 other published proxies, and all 4 had a MWP, 5 including his.
Which also refutes your “Colleagues that agree ” remark.
Now, do you have any published critique of his methods, or is it just your poorly formed opinion?
His proxy also used techniques nearly 20 years old, and well accepted in the literature.
Re Hansen, what was it Korhola said is needed in addition?
Enlighten me, Steve. All I heard was “We need people like Jim Hansen.”
I take your inability to hear the rest of Korhola’s remark about Hansen as a proxy for the care with which you approached his other claims. Otherwise, I can only agree with you that Korhola thinks he’s entirely right.
Steve Bloom: Your
I take your inability to hear the rest of Korhola’s remark about Hansen as a proxy for the care with which you approached his other claims.
Interesting. You ignore my quotations from Korhola’s presentation; and my use of Korhola’s references in the literature (including correct spelling of names and inclusion of dates published), and instead zero in on an imagined slight of Hansen to illustrate “the care” I used on his other points.
Korhola was discussing Foust et al 2009, in the context of “Apocalyptic framing of global warming”.
Korhola said this about Hansen: “we need people like Jim Hansen.”
Immediately after, Korhola said “we need other opinions and honesty”.
So, do you think (choose all best answers) :
a) Foust et al was wrong?
b) We don’t need people like Jim Hansen?
c) We don’t need other opinions and honesty in the global warming debate? Only Jim Hansen’s opinion is needed?
For bonus points, please tell us if you think Hansen has, or has not, made apocalyptic statements regarding AGW.
(hint: death trains)
Otherwise, I can only agree with you that Korhola thinks he’s entirely right.
Actually, I never said or implied this. A good scientist should be confident in his work, but I reserve judgment on the veracity of Korhola’s work. I’ll let his peers comment on that.
your
Korhola is an interesting case. As best I can tell, he’s boxed himself into a minority contrarian position in the recent-past paleoclimate community by defending his own peat bog-based dating research that finds late-Holocene local temperatures in Finland at odds with other work. Most everyone else seems to think his results are crap.
So far, you have not be able to prove or show any of the above statements to be true. I have shown, from your reference, that most of your statements disagree with the literature.
If we distill everything you posted on this forum, your dislike of Korhola and his science boils down to an imaginary slight of Hansen in a presentation to a university gathering.
If I was cynical, I would say that you have an unhealthy fascination with Hansen.
I do agree with Roger. Unless you have something else to give us, you owe Korhola an apology.
Sorry, les, you’re just not good enough at the whole twisting-meaning thing to make this exchange interesting on that level, and as you’re not at all interested in the science I’ll just end there.
Steve: your
Sorry, les, you’re just not good enough at the whole twisting-meaning thing to make this exchange interesting on that level,
I was not trying to twist the meaning. You were, though, and failed miserably.
and as you’re not at all interested in the science I’ll just end there.
I am interested in the science. Unfortunately, you would not, or could not, produce any science to back your view point. In fact, the only evidence you did produce, contained references that refuted your contentions.
I was not expecting an apology to Korhola. An apology would require a sense of honour, strength of character and integrity. As I said, I was not expecting any.
Please cancel my Sierra Club subscrpition.
How can you determine whether “peer review” works or not without an agreed to definition and a determination if what was done in any particular paper or disciple to discern if that review met the criteria? For an absurd case: If by “peer” we mean equals then would you say a trial of a know criminal should have a jury of other known criminals?
So for peer review should the reviewers be allowed to be colleagues and co-authors of other works of the paper’s authors, as Wegman found in his analysis of Mann’s paper?
The only way the incestuous relationship was exposed was through an outside agency (M&M), whereas the gatekeepers (journals) were complicit in the attempt to suppress outside review and the visibility that they hypocritically espoused.
“Closed shops” seem to invite this kind of corruption of values.
Scientific peer review is not broken structurally. It just doesn’t converge on truth quickly enough to meet policy needs in a timely manner. In the rush to fill policy needs with scientific data, critical uncertainties are suppressed or ignored. Eventually, it all works out. But in the process there is always some collateral damage – contituents that feel under-represented. Unfortunately, some teams are expert at averting the Great Attractor Truth.
Mann et al. have already given you the answer: the sign doesn’t matter.
Over at Roger’s blog, AndrewT has tried to explain this to Roger, alas with no success.
But I agree this is a relatively simple matter that certainly should be resolvable. You are, I think, in a position to find yourself an unbiased mathematician or statistician who could be made to understand the issues. You should do that.
Mann’s “answer” is not wrong: the sign on a variable going into a “multivariate regression” doesn’t matter. However the statement he’s made here does not squarely address the issue.
.
Let me explain. The substantive issue is how does Mann’s code treat a proxy when its relationship with temperature changes as you move from the calibration phase into the reeconstruction phase. With a reliable proxy you never expect such a change. With the Tiljander lake sediment series, there is such a change. The consequence is that the proxy’s sense is decided in the calibration step (it’s incorrectly assumed to be positive, opposite to the sense proposed by Tiljander herself) and then carried over to the reconstruction phase, where the MWP is incorrectly cooled. Mann didn’t flip anything “up-side down”, the software did it for him. But, not surprisingly, the cause of the flipping is quite tricky to diagnose.
.
Thus you can see that this is not a simple “multivariate regression” problem. It’s a calibration-reconstruction problem. And it’s not a statistical issue; it’s a coding issue. Mann’s statement strikes obliquely at the problem.
.
Mann should have investigated more thoroughly once he’d seen the McIntyre complaint. McIntyre is not often wrong when it comes to such issues. Not when he’s taken the time to write a formal reply to the journal. As a non-specialist he often expresses himself differently, using colloquial phrasings instead of indsutry jargon. So it comes across as amateur. But that doesn’t mean his arguments are incorrect.
.
You can’t judge a book by its cover.
.
I hope this helps clarify why peer review in this case didn’t catch the error in time to keep it out of the blogosphere.
.
Charity to all.
The substantive issue is how does Mann’s code treat a proxy when its relationship with temperature changes as you move from the calibration phase into the reeconstruction phase. Aha! Thank you. This is the first time someone has made a coherent arguement over this (perhaps it has been said before but lost in the noise, if so my apologies for missing it).
I would answer that such a proxy is simply useless. Getting the sign of the overall series right would not make it useful. A proxy with the properties you describe should not be used.
And it’s not a statistical issue; it’s a coding issue I disagree. It is a data-source issue.
Mann should have investigated more thoroughly once he’d seen the McIntyre complaint – not sure about that. McI’s complaint (assuming we’re talking about the same text) was: “Their non-dendro network uses some data with the axes upside down, e.g., Korttajarvi sediments, which are also compromised by agricultural impact (M. Tiljander, personal communication), and uses data not qualified as temperature proxies (e.g., speleothem δ13C). ” If he meant what you said, he could and should have said so.
Incidentally, “which are also compromised by agricultural impact (M. Tiljander, personal communication)” is an interesting phrase – this appears to imply that the compromise wasn’t clear without pers comm.
William, why do you try to complicate a simple matter?
It is clear that major part of Mann’s replications needs to be recalculated. For starters, with the correct interpretation (sign flipped from Mann’s orientation; alternatively you change the Tiljander series to a category for which negative one-sided test is applied) Tiljander series do not pass the screening. I’m positive you are clever enough to check that from the code.
Since also with the correct sign, it seems that even without screening Tiljander proxies do not enter to CPS reconstructions, the situation for Mann et al should be rather easy: just drop the Tiljander series out of the proxy set. They claim in the SI that Tiljander series “do not matter” to their conclusions anyhow in the first place, why wouldn’t they then simply drop them? They could easily say that because of the controversal nature of Tiljander series, they have decided to drop them out all together and thereby avoid discussing the “sign issue” (save face) and even avoid attributing anything to McIntyre if that’s the real problem here.
> William, why do you try to complicate a simple matter?
I don’t. From my point of view, the “simple matter” was: is the multivariate technique sensitive to the sign of the input proxy.
Since we both agree that it isn’t, the matter is solved. Yes?
At this point, I would say that PNAS’s peer review has not been cast in a good light by the controversies that have developed around Mann et al (2008).
Bender is certainly correct in his description of the problem with the Lake Korttajarvi varve series. Tiljander cautioned that they are unreliable proxies for climate after c. 1720, due to nearby human activity. This was discussed by Mann et al (2008) (link to article & Supporting Information at PNAS.org), see SI pg. 2 column 2, “Potential data quality problems”. Figure S8a is presented to assure readers that this possible source of error is not a problem (I’ll return to this point).
The Tiljander team studied varves (lakebed sediments with repeating annual pattern akin to tree rings) because they are potential proxies for climate. Cooler years in that part of Finland can lead to more snowpack, thus more runoff, thus more fine sediment brought into the lake, thus thicker, more mineral-rich varves. Warmer years produce thinner varves with a higher proportion of organic material.
Local human activity since 1720 has increased the mineral burden in the lake water, and thus caused varves to be thicker and more mineral-rich than would otherwise be the case.
Mann et al’s CPS algorithm looked for a correlation in this proxy between local temperature and varve characteristics during their chosen calibration period, 1850-1995. From reading this precis, you know what the CPS program found: over that time, rising temperatures correlated with thicker, more mineral-rich varves.
Once this correlation was established for 1850-1995, it was extended backwards to the study period, beginning in 200 AD (Fig. 2). This is the contribution of the Lake Korttajarvi varve proxy to the Mann et al CPS-based reconstruction.
Note that sign does matter! The Lake Korttajarvi varve proxy is worse than nothing at all. In years (decades) when varves are thick and mineral-rich, the CPS-derived proxy contributes “warm” to the Temperature Anomaly reconstruction, when Tiljander is clear that the correct contribution should be “cold”. And vice-versa.
Back to Figure S8A and the shortcomings of PNAS’ peer review on this subject.
This Figure shows the full Northern Hemisphere Temperature Anomaly reconstructed by CPS methods for 200 AD through 2000 AD. The green line is the reconstruction as calculated for the original proxy dataset. The black line is the result if the four Lake Korttajarvi varve proxies and three other potentially-faulty proxies are excluded from the dataset.
The black line is supeimposable over the green line. From this, Mann concludes that either (1) the potentially-problematic proxies are not actually faulty, or (2) to the extent that they are faulty, the strength of the analysis is such that they don’t appreciably distort the final reconstruction.
(I’ll note in passing that the three other proxies that are excluded are listed as Benson, Isdale, and McCulloch. In Dataset S1, the latter two are identified as being from the Great Barrier Reef, and thus have no place in a Northern Hemisphere reproduction to begin with.)
An obvious question on Fig. S8A for authors, reviewers, and readers is: What proxies were used in the CPS analysis that yielded the green line? As an outsider who has thought about this for a day or so by now, my answer is, “I’m not sure.” The legend says “the full global proxy network,” but what does that mean? At first, I thought this referred to the full listing of the fifteen screen-passed super-long duration northern hemisphere proxies that are plotted in Fig. S9. That would make sense–check the problematic proxies by comparing the reconstructions derived from the full set of 15 (green line) to the 8 proxies that remain after the 7 Doubtfuls are removed (black line). But, I now believe that “full global proxy network” refers to the hundreds of proxies depicted in Fig. 1, and cataloged in Dataset S1.
This is a critically important point in evaluating the robustness of the CPS Temperature Anomaly reconstruction, which is at the heart of the paper. Did the peer reviewers recognize the issue?
I think the answer is No. For one thing, the source of the data could easily have been stated in the Methods or in the figure legend. If it’s there, it’s buried.
Now that we know that the four Lake Korttajarvi varve proxies were employed by Mann’s CPS so as to add actively-bad temperature information to the reconstruction, what about Figure S8a? The black and green curves are identical. The figure misleadingly indicates that there’s nothing wrong with the proxies or analysis.
One defense is that “the rest of the proxies are so excellent that a few rotten apples changes nothing.” If the green line is a composite of hundreds of proxies that are all weighted equally, then of course the effect of the upside-down varve proxies will be lost. This is trivial and meaningless, except to the extent it is misleading to the naive reader. Such a result contributes nothing to an evaluation of the robustness of the reconstruction.
On the other hand, the green line might be a composite of only the 15 screen-passed super-long duration northern hemisphere proxies (as I originally thought, and as is implied by Fig. S9). In that case, the Mann et al CPS reconstruction is in real trouble! Take away four proxies that feed actively-wrong data into the model (and three others), and discover that analysis of the eight remaining proxies yields an identical temperature anomaly curve?! This defies common sense, and cries out for an explanation.
This is the part of Mann et al that I’ve thought about most carefully. Considering what I found, I am very disappointed in what the peer-review process yielded as a quality-control mechanism.
Dr. Connolley you are near the point of being willfully obtuse. Hopefully the comment here by AMac helps set the record, and you, straight.
Dr. Connolley asserts that the Tiljander proxy should not be used because of its perverse nature as one moves from the calibration phase to the recosntruction phase. Indeed, if you bother to read Tiljander’s work, they do not do a formal reconstruction precisely because of the modern-era contamination. But that does not mean that the uncontaminated portion is useless. In fact, this is precisely what Kaufman et al. (2009) did – they removed the comntaminated bit and proceded with the uncontaminated portion (upside-down, unfortuantely). But happily they corrected this in their Corrigendum.
.
Two points follow, under the Connolley suggestion that the Tiljander series not be used. First, Mann et al (2008) would need to redo their sensitivity analysis, showing what happens when the California pines are removed after Tiljander has been deleted. Second, Kaufman et al. (2009) would have to delete that series and re-do their sensitivity analyses. I can pretty much guess what the result would be in both cases: broken hockey sticks. Good?
Dr. Connolley asserts that the Tiljander proxy should not be used because of its perverse nature
No. Clearly, I didn’t say that. Why are you making things up?
Perhaps it was this comment of William’s that bender is referring to:
[If the proxy represents non-climate information then it shouldn’t be used. But I’ve already said that, so I’m not sure why you want me to say it again. But now I have – are you hap-hap-happy? -W]
http://scienceblogs.com/stoat/2009/10/oh_dear_oh_dear_oh_dear_oh_dea.php#comment-2032693
I too would interpret that to mean “Dr. Connolley asserts that the Tiljander proxy should not be used because of its perverse nature”
What then did you mean?
I’m not “making things up”. I’m insulted by that insuation. If I’ve misinterpreted something, I sincerely apologize. Please do correct me, and we’ll revisit the question.
Yes, Roger that is the precise line I had in mind. But regardless WHO said it, the substantive point is: if Tiljander is forced out, then any leave-one-out sensitivity tests need to be re-done.
.
Why am I making this up? Because I know how to do a proper sensitivity test and I insist it be done properly. Why? Because good science is in my nature.
Connolley, here, clearly leaps to a conclusion before looking at the details of the case. Why? What is the role of faith in his acceptance and defense of Mann’s position?
Steve Bloom wrote: “Korhola’s ideas are way outside of the mainstream, although he’s by no means a denialist.”
It appears that the “mainstream” is fast becoming a small tributary wandering off to nowhere from a deep, wide river.
This is what I find most offensive about the alarmists: their demands for ideological purity.
Wow, wonderful weblog structure! How lengthy have you ever been blogging for?
you make blogging look easy. The full look of
your website is excellent, let alone the content material!
I am not smart enough for all this sophistry, this case is simple:
Sorry for the embedding, I posted a link.
I see William is mistaken that you are female (see his post) – I believe the “cruel mistress” alludes to our mother earth, no?
Ben is obviously a cross-poster.
TH: hey, you can comment over at mine. No, I know full well he is male, like most people called Ben: but in his blog persona as CM is female. But ore interestingly, are you going to risk a comment on the substance?
The best way to introduce others to peer review is to point them to Climate of the Past, and other journals that have open review. Click on any issue, then on any article, and you will see the manuscripts, the referees comments and the replies.
This is the future.
[…] of the journals in extracting the admission from Kaufman. This point was misconstrued by Ben Hale here who interpreted Roger’s post as evidence that the Kaufman error had been detected and […]
A lot of the nay-sayers seem to think that peer review is broken because a few mistakes slip by the reviewers of single paper (or a number of papers). I’m afraid that’s not the case at all – such mistakes are inevitable, but that does not mean peer review doesn’t work.
What is also inevitable is that all mistakes will out in the end. If there’s a counter-argument that is VALID, that counter-argument will also pass the peer review process and the debate will proceed; more slowly than on a blog for sure, but without the 99.9% chaff (unverified claims, bad science, etc.) that appear in the blogs.
The global warming deniers love to pick nits here and there in the literature that supports AGW and think that they’ve somehow disproved AGW. If there were scientifically valid models that account for the the temperature data that we have yet do not invoke AGW, or a scientifically valid processing of the data that does not show a temperature increase in the later part of the 20’th century, then those would pass peer review too. They really would! Peer review is not a liberal conspiracy to keep you all out of the journals – peer review is there to check methods, logic, etc., not ideology. Reviewers aren’t perfect, so some mistakes will inevitably slip by.
If there really were scientifically reasonable competing theories to AGW, there would be tons of articles in the peer reviewed literature describing those theories. There are exactly none. Employ Occam’s razor to determine why. Is it a liberal conspiracy or the lack of a scientifically valid competing theory?
Finding a flaw here or there in some piece of data or some bit of processing, that when corrected for does not even change the result in a non-negligible way does not disprove the theory of AGW. If you have a competing theory that is (or is even close to) scientifically valid, you will have no problem publishing it in a peer reviewed journal. I’d love to see it! Really! It would be fantastic if AGW really didn’t exist or really was negligible.
JJ, you can’t be selective with your science. All nits must be picked without prejudice, and with equal rigour.
You can’t afford to lower standards selectivity in certain areas simply because they conform to political expediency or pre-existing of standards truthiness. It stops being science.
So why not join people here in picking nits – your contribution may yet be a positive one, rather than an antI-scientific one.
May the best science win. Right?
By all means, pick all the nits you want. I have nothing against that. It’s good for the science. But the rhetoric of the deniers always seems to be along the lines of “Aha, we found this error in your work, ergo AGW doesn’t exist”. That’s incredibly poor logic, especially, when the corrected error does not significantly affect the outcome – but they don’t want to worry about that.
If there was a sound scientific theory that explained the data without AGW, it would be properly published all over the place and the AGW proponents would be looking for nits to pick in it. Sadly, there is no such theory.
Well, picking nits is as political as anything else. I can tell you with reasonable certainty that the scientific literature on the hibernation practices of mole rats is likely to have many more unpicked nits than the scientific literature on vaccinations or on climate change. That’s just a fact about contentious issues that carry with them possible policy implications. When the policy implications are heavier, people take out the magnifying glass. Maybe that’s as it should be.
For denialists, the nit selection criteria are all about the politics, not the science, and I think failing to take that into account whenever considering them is missing the key point. They attack the science solely because they dislike its policy implications. When scientists pick nits (and they do it all the time) all sorts of non-scientific criteria may be involved, e.g. professional jealousy, but generally they’re not primary. A scientific career could not be built on e.g. what McIntyre does.
Denialists could acquire a kind of credibility if they focused on the aspects of the science that form the key underpinnings of policy. Instead, they pick on tangential or minor issues such as the amplitude of late-Holocene climate change and the modern surface record, and try to argue that the claimed flaws must mean that the rest of the science is questionable. After a while it just gets boring.
Steve Bloom, October 31, 2009 at 12:45 pm —
> For denialists, the nit selection criteria are all about the politics, not the science… and [denialists] try to argue that the claimed flaws must mean that the rest of the science is questionable. After a while it just gets boring.
At Stoat, I posed five claims about the Tiljander varve proxies, and their use by Mann et al (2008). I’m not a “denialist” (I don’t think), and the Tiljander proxy selection criteria isn’t about politics (I don’t think). That I conclude that Mann et al’s use of these proxies was flawed doesn’t mean that the rest of paleoclimatology is questionable (I don’t think… well, all science is “questionable,” no? So it doesn’t mean that the rest is wrong).
For these reasons, you might find the discussion at Stoat interesting rather than boring. Since you are both knowledgeable on paleoclimate reconstruction and well-disposed to many “mainstream” approaches to it, I suspect W. Connolley would welcome your input in the “Tiljander” thread. I’m interested in your opinion on my responses to W. Connolley, here and in the following comment.
[…] of the journals in extracting the admission from Kaufman. This point was misconstrued by Ben Hale here who interpreted Roger’s post as evidence that the Kaufman error had been detected and […]
[…] of the journals in extracting the admission from Kaufman. This point was misconstrued by Ben Hale here who interpreted Roger’s post as evidence that the Kaufman error had been detected and […]
Wow, superb blog format! How lengthy have you ever been blogging for?
you made blogging glance easy. The total glance of your web site is magnificent, let alone the content material!