h1

Hockey Stickler

October 5, 2009

Roger has assured me that if I include the words “hockey stick” in any post, I will generate traffic.  At this point, I’m not sure if I really want that traffic, as I’m relatively new to this whole thing and I still have a little boy who seems to want me to give him fatherly attention.  (Audacious, I know.  Any other three-year-old and I wouldn’t give him the time of day, but this kid manages to persuade me to plop my fat ass on an undersized wooden chair and add color to the outlines of obscure Candyland characters.  I also have significant publication responsibilities…but never you mind, this whole discussion has been heaps of fun, so I think I’ll continue.)

In other news, Maurizio has given me honorary props on his blog, so I guess I owe him thanks for the extra traffic.  I want to take up his issue as well.  Not right here though.  Eventually.  These things must be spaced out.

One problem that seems to be evading commenters is that I’m not qualified to make judgments about what has been scientifically demonstrated.  I simply don’t have the climate background and I rely very heavily on people I deem to be reliable sources.  As it happens, most people are excluded from this category of reliable sources, though those excluded from the category are not distinguished by whether they are “skeptics” or “proponents.”  It is clear that many proponents don’t have anything like the scientific background required to make a justified assessment of the strength of a given scientific argument; and it is equally clear that some skeptics do have that scientific background.  What is also clear is that the climate data span an incredible range of scientific subfields, so I would be shocked and surprised if any single person, even Rajendra Pachauri, is qualified to judge all of the science all of the time.

As a lowly philosopher, I’m only qualified to judge whether something has not been demonstrated, and even my judgment on that score is questionable.  If I can think of any plausible reason why something has not been demonstrated by a given argument, and provided that the burden of demonstration is possible to meet, then it seems reasonable to me to say that the claim in question hasn’t been demonstrated.  The plausible hypothetical objection (emphasis on ‘plausible’), in other words, should serve as enough reason to say that some burden of demonstration hasn’t been met.

For instance…

For instance, one objection that keeps popping up is that Briffa’s sample is too small.  That’s really a question for dendrochronologists to answer; not for dendro-dabblers, like me.  I’ll tell you honestly, it seems small to me, but I’m not a dendrochronologist.  Counting tree rings has for me all the enticement of counting holes in ceiling tiles, so I’ve never bothered to explore it.  I’m not competent to judge.

But here’s the other thing.  I do know that statisticians and other scientists sometimes have neat little tools for handling small sample sizes.  So again, even though the sample size seems small to me, this is a charge that needs to be addressed and evaluated by people who understand the statistics and the science, not by me nor by anyone else who lacks the technical background, both in statistics and in dendochronology.  Fortunately for all of us, we have the very flawed process of scientific peer review, which is what I’ve been pressing people to wait for. Yeah, it’s frustrating and not entirely impartial.  Peer reviewers are political actors, after all, just like everyone else.  But that doesn’t mean that it’s not a reasonably good system for distinguishing better from worse arguments.

There’s really more to say on this, so I hope to come back to it later too.  In the meantime, I want to get to one proposition raised by Bender.  He’s antsy for a reply, and his antsiness counts as a reason for me.  Bender writes this in the comments:

Perhaps Ben Hale, as a philosopher, can give us his take on IP claims on data and computer code used in support of public policy?
.
Example. If I were a chemical company wanting to put a factory in Ben’s backyard and he wasn’t satisfied with my claims of safety, am I obliged to disclose to him the details of the chemical process, plant engineering design, etc? Or can I dodge the issue by hiding behind the claim that the design of my plant is a valuable “trade secret”? Does my right to profit trump his right to safety?

I really like this comment.  These are the kinds of things I can deal with, and we routinely deal with such cases in philosophy.

Let’s see:

If Ajax Chemical (A) wants to put a chemical plant in Ben’s (B’s) backyard, and B is not satisfied with A’s claims of safety, we need quite a bit more information about what is going on with A’s claims of safety and with B’s concern.  I don’t think A is ethically required to disclose to B the details of the chemical process, plant engineering design, and so on, just because B is worried about them.  I do, however, think that A is ethically required to demonstrate that A’s plant is safe (or to work out some other arrangement with B such that the dangers imposed by A’s plant are satisfactorily mitigated or avoided).  In other words, neither B’s concern nor A’s pronouncement is doing the work.

Here’s why: A could be lying or B could be crazy.  Or both.  Or some similar variant: A could be crazy, B could be obstructionist, A and B could be fighting a proxy battle over C, and so on.  We don’t know.  We can assess either A’s or B’s claims, however, based on their fit with what is right, what is true, and what is truthful.

I also said, however, that I do think that A is required to demonstrate that A’s plant is safe, or to work something else out with B.  That obligation stems from B’s baseline state, which is by assumption a life in which A does not exist near enough by to screw things up for B.

(Note please that I’m not talking about legal requirements and I’m not talking about political requirements.  I’m talking about ethical or moral requirements.  They’re related, but let’s all admit that laws can be terribly wrong and bad; and politics is, well, politics.  Also, note that it matters to me what falls after the “because.”  I want to know the reason that A is required to demonstrate that his plant is safe.)

What Bender seems to want me to buy is that if there is some identifiable public health risk to a population B from the pollution of a plant like A, then even given concerns about the damage to private property or to proprietary information, A has a prima facie requirement to hand over its data.  I will grant, as I said, that that proposition is true, but what Bender doesn’t seem to acknowledge, and what his example emphasizes, is that the gap between an identifiable public health risk and a perceived public health risk is wide indeed.  Just look at all of the fuss over childhood vaccinations, fluoridation of water, spraying of mosquitoes, massive cattle slaughters in fear of e-coli, and on and on.

So we really need a mechanism to distinguish between the perceived public health risk and the identifiable or “real” public health risk.  Usually this mechanism is something like the scientific method, and every institution has, more or less, procedures in place to address public concerns of the nature that Bender notes, some being quite a bit faster than others.

Which brings us back around, finally, to the nine year question (9YQ) — about what took Briffa so damned long to release his data to the scientific public — which I assume is what motivated Bender’s question in the first place.

If it is true that there is a mile-wide gap between the perceived and the real, and if it is also true that any old nincompoop can raise an objection, this seems like a mighty unpleasant way of bridging the gap.  There are, as I said, lots of crazy Bs in the world.

I’m not at all saying that McIntyre is crazy, incidentally.  Nor am I saying that crazy Bs can’t raise perfectly legitimate objections.  What I am saying is that each objection has to be assessed as relevant and posing a serious challenge to any given scientific (or non-scientific) claim.  Once it is established that that objection is relevant and posing a serious challenge, it is then, and only then, that the prima facie requirement to disclose all relevant data, regardless of proprietary information or intellectual property rights, kicks in.

For the time being, so far as I can tell, McIntyre’s objection has not been demonstrated as relevant or posing a big enough challenge to Briffa’s findings… though that could easily change.  I simply don’t know and I can’t judge.  That is clearly (clearly!) what many of you readers are trying to claim.  Moreover, it’s not clear to me that McIntyre’s objection had enough scientific standing, or could establish enough standing, to demonstrate this relevance earlier, thereby warranting, straight out of the gate, the release of Briffa’s data.  History is relevant here, so I’m happy to be corrected on the facts of the case.  Some folks have called attention to the journalistic standards of Science… and if it is true that all data, even tangentially related data, must be unleashed to an adoring public, no matter the legitimacy of their concern, then so be it.  But I find it hard to believe that there aren’t some other established conventions that speak against a widespread scatter-shot release of all data.

As I mentioned, the mechanisms may be different for climate science versus some other science like, say, the medical or epidemiological science involved in a swine flu scare.  In the case of the swine flu scare, there’s good reason for the CDC to move very, very quickly, because every sneeze is a potential death tornado.  In the case of climate science, by contrast, at least until very recently every tornado was a mere sneeze, a mere tickle, in the overall climate policy picture.  The public could wait for the slow gears of the science establishment to run all challenges through peer review.  Not so with the dreaded swine.  One sneeze and you’re surrounded by zombies.

So maybe what some are calling for is a change in the mechanism by which the climate science establishment identifies problems in the science, given urgency and cost of the proposed solutions.  I think that’s an open and interesting question — in other words, how does the climate policy establishment adjust the climate science standard of review to better meet the shifting needs of the public — but this, alas, is a topic that will have to wait for another post.  I’ve gone on for far too long as it is.

Advertisements

17 comments

  1. Michael Berube, hockey blogger


  2. Ben, oh Ben! Would you have ever believed yourself arguing against the public sharing of data everybody seems so keen about, including “Science”, NAS and NASA?

    Public sharing to all, not just to nincompoops or

    The objection is relevant and posing a serious challenge argument is rather weak. One could respond that no objection can be established as relevant and/or posing any serious challenge if there is no access to all relevant data (incidentally, you are assuming that McIntyre wanted to object to and challenge Briffa’s results…even if that were true, how could he proceed without trying to replicate Briffa’s first? and how could he replicate Briffa’s without access to all relevant data?

    I find it hard to believe that there aren’t some other established conventions


    This should be easy to resolve without having to hide behind don’t know and can’t judge. For example the data publication policy for “Science” should be easy to read for anybody that will ever think of publishing an article on “Science”. If it is not, then there is a clear failure on the part of the magazine (i.e. every author would need access to a lawyer to understand Science’s publication policy).

    If it is, then I am sure you yourself will be able to publish an exegesis of that text, verifying if it is compatible or not to the withholding of data by any article’s authors. Many people claim it is not.


  3. Another point. There is one big problem with the “Ajax” analogy.

    No serious Company will ever want others to be able to replicate its capabilities.

    No serious Scientist will ever want other not to be able to replicate their findings.

    Told otherwise: There is no business if people can just replicate your business. There is no science if people cannot just replicate your science.

    So what can we think of instead? Imagine being told by a doctor (A) that, according to their data (D), you need to undergo surgery or spend enormous amounts of money on pills for years to come.

    Now, do you have a moral right to get a second opinion from a doctor B? And if you do, would it be ethically acceptable for A to withhold D from B?


  4. Ben, I know you’re new to this so we’re all happy to cut you some slack. Can you be unaware that the established procedures for peer-reviewed publication include disclosure?

    The bottom line: real science is falsifiable and replicable. Journals support that through disclosure / transparency policies.

    In bender’s scenario, does your answer change if Ajax Chemical is required to publicly disclose the details of its plant safety mechanisms in any building permit / plan application?

    The 9YQ is a question of failure to enforce or adhere to existing policies.


  5. Oh, silly rabbits, you won’t get off that easily. I may be new to this, but I ain’t naive.

    Pete, I’m suspicious about your falsifiability claim. A real scientific theory may be, in principle, falsifiable (though even that is questionable, cf. Duhem, Quine, and Hempel), but I’ll make an even more pragmatic point. It’s not clear at all from what’s been going on that Briffa’s conclusions are neither falsifiable nor replicable. Just because the data hasn’t been made available doesn’t say anything about the falsifiability of the finding.

    Here’s another scientific claim: that Jupiter’s moon Io has a core of green cheese. That’s a scientific claim that is falsifiable…. and yet, we have no evidence to falsify this claim. It’s not that it couldn’t be easily falsified if we had that evidence, it’s just that we don’t have the evidence in our hands.

    Oh, and incidentally, as for the compliance policies of _Science_, here they are, taken right off their website. Maybe they’re spelled out in depth somewhere else, but this is what I read… and this is certainly not carte blanche to release all data:

    “Data availability: After publication, all data necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science. We recognize that discipline-specific conventions or special circumstances may occasionally apply, and we will consider these in negotiating compliance with requests. Any concerns about your ability to meet Science’s requirements must be disclosed and discussed with an editor. For further information about accessibility of data and materials, see the following resources.”

    http://www.sciencemag.org/about/authors/prep/gen_info.dtl#dataavail

    If the original Briffa paper was published in a different Science imprint, I’m happy to parse that language too.


    • About Io: I am sorry but it’s exactly the wrong example :).

      Io is the one satellite whose interior is being spewed out, and lots of it, so there’s plenty of chance to check what it is made of (=plenty of evidence). And there’s no indication of green-cheese-related chemicals in any of the volcanoes.

      Perhaps you might use Callisto next time, it’s got one of the oldest surfaces in the whole Solar System and models of its interior are speculation based on gravimetric data.

      About Science: which part of “all” (as in “all data necessary etc etc”) do you want to disagree about, especially in the context of McIntyre looking to understand/assess Briffa’s work?


    • I agree, not “all” data. However, isn’t it obvious from the rapid replication and subsequent vigorous discussion, that Briffa’s 2000 publication was not “understood” or “assessed” until now?

      You ask a good question about green cheese in Io.

      Your statement: “That’s a scientific claim that is falsifiable.”

      I would say that will become a scientific claim once it is falsifiable. For now, it is simply conjecture. A scientific hypothesis must be testable, or it is not science.

      If my science work cannot be tested, i.e. potentially falsified, then it is not science.

      A personal, hopefully helpful example follows:

      I have an interesting medical condition that provides some insight to this. For several years, I have had a sleep disorder that is otherwise unknown to science (last time I checked a few months ago.) Among other interesting aspects arises this set of facts (an approximation, I don’t have the details handy):

      * In men, 75% of HGH (Human Growth Hormone) is generated during the Delta sleep phases (3,4 of 5 total phases).
      * I am missing those sleep phases.
      * Nobody knows if HGH drives delta sleep or the other way around. Correlation is known, causation not known.
      * My HGH level cannot be accurately measured.
      * It is possible that my lack of delta sleep means I am rapidly (!) aging, due to lack of HGH.

      This is a potentially alarming question. Am I at risk or not? I took this to the highest levels of the AMA. The answer: sorry, we don’t know and can’t figure it out. No use worrying about it. Someday maybe we’ll learn more.

      The question of causal relationship between HGH and Delta sleep is a FUTURE scientific question. Today, it’s just a curiosity.

      What say you?


      • MrPete: Sorry to hear about what’s happening with you. I’m not an endocrinologist, and I have just enough knowledge to be particularly dangerous (I have medical background and a family member with an interesting series of endocrine issues).

        My (albeit limited) understanding was that that GH isn’t easy to measure but can be done. Often, people use measurements of IGF and type 3 pro collagen as proxies for HGH (another proxy study!!!). In any case, there’s a lot of variability in measurement of both, partly because of differences in assays, partly because of the nature of GH and the proxies (the levels in the body constantly change over time).

        However, if you meant that you have no measurable GH after reasonable attempts to assess the GH levels (either directly or indirectly via IGF), then GH replacement therapy would be a reasonable consideration. There’s a good deal of data on folks with pituitary problems, traumatic brain injury, or post surgical causes of GH deficiency that show improved outcomes when GH replacement therapy is used.

        If you have not already done so, would suggest getting in touch with another endocrinologist, preferably a University based endocrinologist doing research into GH issues if you aren’t getting answers.

        Good luck!

        Bruce


  6. Ah, but you’re being too scientific. Ignore what you know. Use your imagination. Green cheese could easily be in the core of Io even with your supposedly clear evidence to the contrary. You have no idea if it’s the core of Io that is being spewed out or if there is an outrageously intelligent super-monkey with his opposable thumb on the cheese nozzle keeping the gooey loveliness from spewing into the atmosphere.

    Occam’s razor not withstanding, your evidence alone isn’t enough to falsify my claim.


    • Thank you very much Ben. That’s then where I had left my outrageously intelligent super-monkey with his opposable thumb on the cheese nozzle keeping the gooey loveliness from spewing into the atmosphere!!

      I have been looking for him (in the depths of Triton) for ages!

      Thanks again.


  7. Dear Doctor Hale,

    You state: “Once it is established that that objection is relevant and posing a serious challenge, it is then, and only then, that the prima facie requirement to disclose all relevant data, regardless of proprietary information or intellectual property rights, kicks in.”

    Your suggested process ignores the undisputed fact that until Biffra released his data last month, the specific statistical objection made by McIntyre to the sample size could not be made because Biffra refused to reveal the size of his sample.

    Extending your Io example; suppose Biffra travels to Io and drills a core of that moon. He then writes a paper which is peer reviewed and published in Nature which concludes Io’s core is made of green cheese. This conclusion contradicts the existing paradigm that Io’s core is made of some other material (just as Biffra’s temperature reconstruction from tree rings contradicted the then existing paradigm that the Medieval Warm Period was warmer than the Current Warm Period).

    Skeptical geologists ask to see his cores and data. Biffra refuses to provide the core and data. Under the process you suggest, how would those skeptical geologists demonstrate that any “objection is relevant and posing a serious challenge” without the data?

    Regards,

    WJR


  8. The problem isn’t just that geologists won’t be able to falsify absurd theories about the cores of planets, but that falsifiability isn’t alone such a great standard. There’s much more that goes into the validation of scientific claims. Occam’s razor, for instance, helps us weed out a bunch of ridiculous garbage, like outrageously smart super-monkeys.

    Problem is, we need to appeal to external reasons to exclude the super-monkey explanation; like that there’s a better, a simpler explanation. More than this, we can’t simply rely on the simplest explanation, because that too could be wrong.

    When you hear hooves, don’t think “Zebras.” But if you hear hooves, don’t necessarily rule out Zebras.


  9. Occam’s razor is just a shortcut.

    I’m an outrageously smart super-monkey. Ridiculous, I know, but can you prove me wrong? 🙂


  10. (Noodling this in the back of my mind while working…)

    I’ll play devil’s advocate w/ myself. I don’t know the answer to the following, but it seems a potential defense for science-as-non-currently-falsifiable.

    Was Einstein’s Relativity falsifiable *at the time* in any way? We know various future tests could be conducted, but what if anything provided then-current support?

    I honestly don’t know.


  11. Well, falsibiability is supposed to offer a test of whether a theory is scientific or not. So, in principle, and being very charitable to Popper, the theory of relativity is falsifiable, even if it’s not the case that we have a way of testing it right now. Its falsifiability isn’t contingent on our instrumentation, but rather on whether such a view could be shown to be false.

    Psychological egoism — the view that attributes egoistic motivations to any intentional action by an actor — is unfalsifiable, since in principle we can always offer an argument that attributes an egoistic motivation back to the actor.


  12. I don’t buy this idea that anything falsifiable *only* in theory is scientific. That does open the barn door to all kinds of craziness.

    Got any Popper experts at hand? Pat Frank who hangs out at CA is apparently one. Glad to ask him to weigh in.


  13. […] and review.  You can read all about it at the three, relatively hot hockey stick threads here, here, and […]



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: