Pete asked my opinion on the methodology on this paper, so here's my
response as a statistician:
Here's my overview: It seems to be a clear, well written paper. There
don't seem to be any abuses of methodology or cherry picking/biased
use of citations. It uses an old, but useful heuristic developed by
the person that first connected smoking with lung cancer and first
implemented a randomized clinical trial (AB Hill). I am no expert in
causality, but it seems to me that there are better tools available
for this, such as "Causal diagrams for epidemiological research"
(Greenland et al), or the work of Pearl, or using a formal statistical
approach to a meta-analysis or a variance components model (my
favorite tool, but when you have a hammer...).
To be more specific, I would like to have seen a formal statistical
analysis instead of a heuristic used. The Hill method is useful, but
it's not part of a bigger paradigm, and the paper itself says that
(top of right column on first page). Even more specifically, there is
no framework, nor probabilities for what an outcome in favor of, or
against causality means. It has been useful in the past, but I think
there are much better tools that have been developed in the last 48
years for better quantifying causation. (anybody want to hire me?)
The outcome is that they are looking at a very specific issue: whether
they think there is causation between a main effect between one of the
three routes bees can be exposed to neonics (diet) and population
declines. It doesn't discuss them as an interaction, nor does it
discuss confounding.
However the lead author did publish such a paper [27], "A
meta-analysis of experiments testing the effects of a neonicotinoid
insecticide (imidacloprid) on honey bees". I skimmed it and it seemed
to be a good read, and some interesting papers cited it:
http://scholar.google.com/scholar?cites=11969851590892635642&as_sdt=5,34&sciodt=0,34&hl=en
Here are some notes I made while reading the paper:
Note on introduction:
"...science-based risk appraisals normally conclude that pesticides
should play a role in crop protection because the benefits of
increased productivity outweigh the risks to non-target organisms.[3]"
I didn't look at [3] (Berenbaum et al), but the concept that
ecosystems have value and provide services in an economic sense is a
newer one. I first learned about this when I was in a postdoc at the
EPA. The phrase is "ecosystems as a service".
http://www.epa.gov/research/ecoscience/eco-services.htm
http://en.wikipedia.org/wiki/Ecosystem_services
If a huge forest in the amazon can turn CO2 into useful metabolites;
or if native pollinators are pollinating commercial crops, their
service isn't free. It has some value if they were to disappear,
somewhere the slack would have to be taken up to avoid a tragedy of
the commons scenario. I think you'll see people using this "ecosystem
as a service" argument more, especially since some of the risk
assessments I see undervalue the "ecosystem" [it's a swamp, it's
filthy vs. the currently land use squesters these nasty compounds,
helps remove these types of compounds from the air/water and the
habitat it provides to local and migrational birds/insects, a service
which would cost $xxx to replace].
The paper says that too, "... [in Europe] it is taken as granted that
a pesticide’s use is unacceptable if it seriously threatens a
non-target species that contributes to human well-being by delivering
an important ecosystem service."
Section 1.1:
It's interesting to read about the guttation problem [37], which is
related to Randy Oliver's comment in his "What happened to the bees
this spring?"(2013) paper in the "Drought stressed plants section"
about how pesticides, specifically systemics are not field tested
during droughts, which could change their dilution factor:
"That said, beekeeper Bret Adee brought an interesting question to my
attention: the dose of seed‐applied systemic insecticides (whether
neonic or other) is based upon the dilution factor as the plant grows,
so that the residues in nectar and pollen will be reduced to below the
“no observed adverse effects level.” But what happens during drought,
when the water‐stressed plants only grow knee high before desperately
flowering? There would be far less plant biomass in which to dilute
the insecticide (assuming that drought‐stressed plants absorb the same
amount from the seed treatment)."
Section 1.2:
This is a very simple model which is not used statistically (there are
no formal estimates of any of the parameters, the b,d,b*,d*), however
it is the basis of their scoring in section 2 (the informally ballpark
the estimates of the parameters, b,d,b*,d* based on the literature).
Section 1.3:
First, I'm not a causation expert. I know enough about structural
equation modeling to be dangerous. I downloaded and read/skimmed
references [43 and 44]. It appears to pass the bar for peer reviewed
research, but it doesn't seem to be hard science to me.
It is not clear why they chose a causality model from the 1960s (5064
citations over 48 years = ~105 citations/year) vs something
contemporary like Pearl's "Causality", 2000, (6478 citations in 13
years, ~500 citations/year), or "Causal diagrams for epidemiologic
research". [just because it's old, doesn't mean it's not good
research; just because something is cited a lot, doesn't mean it's
good research]. Our mathematical notions of causation have changed
alot since then. In the abstract of a paper they cite [44], "The key
analysis tool to assess the applicability of Hill's considerations is
multiple bias modelling (Bayesian methods and Monte Carlo sensitivity
analysis); these methods should be used much more frequently"
Personally, I had a very hard time understand the purpose and
methodology of reference [44]-- it was not clear if it was actually
supporting Hill's paper "the danger of misapplying [the Hill
considerations] can be high", but I haven't read enough of the
causality/philosophy literature to judge.
Section 2:
Here they just try to use the methods described in section 1.3, a
quantified [45] scoring for the Hill criteria. They take a couple
paragraphs to explain the Hill criterion and describe why they gave it
a score based on their ballpark estimates of b,d,b* and d*.
2.1
Here's the type of statement I have issues with:
"Secondly, even if a dietary neonicotinoid caused a laboratory-scale
reduction in colony performance (e.g. in foraging success or
fecundity) under field conditions, it is not clear that this meets the
condition of sufficient harm"
Section 3 (Discussion):
The last paragraph leaves me a little troubled. It describes an
interaction effect between neonics and X, but with no main effect.
3.1:
"it is concluded that trace dietary neonicotinoids are not implicated
in population declines of honey bees. The evaluation is provisional,
however...". I think "it is suggested that" would be more accurate
here.
I strongly agree with their directions 1,2 and 4.
- we need more better data
- we need a computer model, and we need to validate that computer
model (I emailed Mary Myerscough yesterday)
- we should look at interactions more formally
They're more gungho about Hill's criteria than I am, but I'm biased...
It does seem like a useful part of an exploratory data analysis in
epidemiology though.
Some of my observations about the paper:
- I would argue that beekeeping is conducive to designed experiments,
and don't think the analogy between beekeeping and climate change
holds weight (the climate change research is much more technical than
what they present)
- it's not a statistics paper, so
- this is not a meta-analysis in the statistical sense
- there is no statistical model, no statistical hypothesis test
- there's not enough focus on confounding, specifically between:
use of neonics and incidence of Nosema c; migratory beekeeping and
exposure to pesticides; migratory beekeeping and exposure to RNA
viruses (DeRisi paper also on Adee's bees); norther hemisphere
beekeeping and climate change (like the artic oscillation,
http://www.wunderground.com/blog/JeffMasters/comment.html?entrynum=2010
);
- it seems to be some evidence, but it's not definitive/convincing.
- personally, I would have much preferred to look at a variance
components model or a meta-analysis like the first author's other work
A bit of humor about correlation, causation, necessary and sufficient conditions
http://xkcd.com/552/
(the mouse over)
"Correlation doesn't imply causation, but it does waggle its eyebrows
suggestively and gesture furtively while mouthing 'look over there'."
***********************************************
The BEE-L mailing list is powered by L-Soft's renowned
LISTSERV(R) list management software. For more information, go to:
http://www.lsoft.com/LISTSERV-powered.html
|
