Another rainy weekend in Virginia.  This time, no tornadoes.

The discussion of screens and varroa mites has prompted multiple people
to cite various studies.  Each study was tossed on the table in expectation
that it would land with a resounding thump, bearing weighty scientific
evidence in support of a point of view.

But how "weighty" are the papers that are trotted out to influence the consensus
on practices intended to assure the survival of one's colonies?  Are they more
"weighty" than the consensus of the participants in the discussion?  Does it strike
anyone as funny that a beekeeper with decades of experience and hundreds
of hives feels somehow obligated to cite a study of a few dozen hives over
less than a year?

I've been reading the literature for quite some time.  As compared to papers in
other fields, there are significant differences between the types of experiments
and studies that result in published papers of interest to beekeepers, and the
usual pile of published scientific papers from other fields.

My intent is not to "critique" any individuals or papers. My sole aim is to
attempt to point out some of the limitations that researchers are forced to accept,
and suggest an approach that might overcome many of these limitations.

1)  A Small Number Of Publishing Researchers

There are far fewer researchers working in areas related to beekeeping than in
many other fields of scientific inquiry.  To give an idea of the scale of difference,
I could start now reading only newly-arrived pre-prints of papers relating to radio
astronomy, but new e-mail announcements arrive faster than I could read the
referenced papers.  I would never "catch up", even reading 12 hours a day, 7 days
a week.  To be honest, I don't even have the time to read all the abstracts.

Fewer researchers mean that fewer studies are done, and as a result, many
experiments remain unverified (or unchallenged) for years. Some are never
subjected to the acid test of "reproducing the results".  Even studies with
surprising or counterintuitive conclusions can stand unchallenged for years.
Time does not allow, and funding is not available.

2)  Long-Term Studies Are Rarely Done

Many of the problems that beekeepers look to science to solve involve the
survival of their colonies, but many colony losses occur over time spans that
are longer than the length of the majority of the studies. Sadly, researchers
are subject to the tyranny of funding.  Most studies relating to bees have a
term of less than a year.  While I'm sure that there have been many attempts
to obtain funding for longer-term studies, such studies are very rare.

As a result, the majority of knowledge and consensus about long-term effects
tends to based almost entirely on anecdotal reports from beekeepers.  Some
fraction of these anecdotal reports are better described as "apocryphal", and
very little of this type of data ends up being gathered, analyzed, or published.

In the field of medicine, a similar situation exists, where most studies are neither
long-term, nor large-scale.  The difference is that anecdotal reports from doctors
are considered a valid source of valuable data about long-term effects.  Doctors
are viewed as doing a better job of keeping records, perhaps because they use a
more consistent set of terminology and test methodologies than beekeepers,
or perhaps merely because they do keep records.

3)  Most Studies Test Small Numbers Of Colonies

Again, this comes down to funding, but the bottom line is that it is difficult to
interpret data from a small number of colonies, since each hive is a significant
fraction of the "data set".  Most studies that use sample sizes that can only
be described as "minimal", and many are being done at the limits of what would
be considered "acceptable" for publication in other scientific fields.

While large numbers of colonies certainly exist, the larger beekeepers tend
to be migratory, and clearly cannot be expected to willingly allow some number
of their colonies to be "controls" when "control colonies" can be expected to
suffer fates such as lower production or death.

To make matters worse, even if each beekeeper with more than 500 hives
"adopted" a research project, many researchers would still be "orphans".

4)  Outside Factors Can Overpower Experimental Parameters

Everyone has seen side-by-side colonies, managed in exactly the same
manner, with very different production totals.  Most beekeepers try to
eliminate obvious factors from the equation, and are left to shrug and
blame poor production (brood, honey, whatever) on a queen that was
somehow not up to snuff.

But these same variations can skew studies, moreso when the studies
are done on small numbers of hives over short periods of time.  Yes,
the careful application of statistical rigor to one's data can eliminate
much of this problem, but sample sizes used in many studies are
smaller than would be acceptable in other fields, making this a factor
that increases doubt about nearly all studies done.

To make matters worse, only a tiny minority of beekeepers have
sufficient training in statistics to allow them to judge the statistical
merits of one study or another, meaning that all study conclusions
appear to many to be of equal authority and "weight".

For a good overview of some basic concepts of statistical analysis
in plain English, take a look here:

   http://www.sportsci.org/resource/stats/indexalpha.html

Confused by it all?  Don't feel bad.  Keep at it. I've used statistical
software made by the SAS Institute for several decades, and even
this high-end software includes manuals and online help that constantly
attempt to reinforce basic statistical concepts.  Many well-funded
research projects employ a professional graduate-level statistician to
"keep the data honest", and create an aura of respectability similar to
that created by the use of an independent accounting firm's audit on a
company's financial reports.  (Enron/Anderson are proof that numbers
often have sharp edges, even in the hands of professionals.)

5)  We Have Fired All The "Magic Bullets"

Experiment design grows in complexity when one has multiple variables.
Most experiments and studies focus on one variable at a time.

Chemical company promotions to the contrary, there appears to be no
single product or management practice that can assure colony survival.
We are left to attempt to judge the relative merits of different combinations
of tools, techniques, and practices.  This makes for much more complex,
and hence, expensive experiments.

One Possible Future Of Practical Research

The future might lay with the hobbyists.  Groups of hobbyists, properly
briefed and willing to tolerate being "instructed" in technique, could
certainly implement studies designed by researchers.  For example, if 20
beekeepers, each with 10 colonies were willing to participate in a study,
this would reduce the costs of a 200-colony study to no more than the
cost of "design", "data analysis", and a meeting or two to outline the
specifics of the study.

If funding was available, the studies could insure the colonies, by replacing
any/all colony losses with packages.  This would be a very good deal for
the beekeepers, since colony looses are nonzero for nearly everyone
regardless of management practices.  This would also be a very good deal
for the researchers, since the overall cost of such an approach would be less
than doing the same study "on campus", even assuming high losses.

Large scale, long term "experiments" are going on now, but no one is asking
beekeepers to gather data in a consistent manner, and there is limited ability
to assume use of consistent practices suitable for a coherent set of data.

How long will the group as a whole continue to be epistemologically limited in
our ability to search for patterns, but still ignore the fact that nearly every hive
on the planet is being subjected to one experiment or another, intentional or
unintentional?

In other words, when do we get out act together, and start acting in our collective
best interest?  How many hives do you have at risk?  What were your losses
last year?  Can you afford a spiral-bound notebook?  Can researchers afford
to ignore an offer of participation?

        jim