> You refer to "sophisticated statistical methods 
> that can be used in survey analysis." This fails 
> to acknowledge the fact that Garbage In, 
> Garbage Out. No amount of post processing 
> can turn gravel into gold.

If each unqualified declaration made by a beekeeper were backed by a wager
of some small amount of money, I could buy my little sister a Hance 675 with
the proceeds!

A commonly-used technique is to include survey questions to identify those
who might provide "garbage" answers to surveys, and down-factor that cohort
of data in the analysis:

"Predicting Careless Responses and Attrition in Survey Data with
Personality"
Adam W. Meade & Gabriel Pappalardo, NC State
"Virtually nothing is known about the personality of undergraduate
respondents that fail to complete or respond carelessly on surveys.
Conscientiousness, Extraversion, and Agreeableness were related to survey
attrition and carelessness resulting in significant differences in persons
carefully completing the survey and those lost via attrition and careless
response screening"

http://www4.ncsu.edu/~awmeade/Links/Papers/Meade&Pap(2013)Careless&Attrition
.pdf
http://tinyurl.com/pxmu525

One can even account for those who don't even respond to the survey at all!

Lindner, J. R., & Wingenbach, G. J. (2002). Communicating the handling of
nonresponse error in Journal of Extension Research in Brief Articles.
Journal of Extension [On-line], 40(6). Available at:
http://www.joe.org/joe/2002december/rb1.shtml

Lindner, J. R, Murphy, T. H., & Briers, G. E. (2001). Handling non-response
in social science research. Journal of Agricultural Education, 42(4), 43-53.

Miller L. E., & Smith, K. (1983). Handling non-response issues. Journal of
Extension On-line, 21(5). Available at:
http://www.joe.org/joe/1983september/83-5-a7.pdf

> I already quoted Katie Lee, of BIP, who 
> pointed out the shortcomings of the BIP surveys.

Yes, everyone in the business of responding to inquiries from beekeepers is
quick to qualify their assertions, as they don't have time to answer a
tedious critique from someone not selected as a referee by a journal, but at
the same time do not want to insult anyone.   But this does not mean that
people publish one thing after peer review, but admit to having overstated
their case under the withering cross-examination of a beekeeper.  Everyone
will certainly try as hard as they can to find a basis for agreement and
common ground with any critic, as the critic then goes away vindicated, and
the interaction ends sooner than it otherwise might.

But this does not mean that survey data cannot be better-analyzed where it
was not, or that surveys cannot be better-designed to be powerful
data-collection tools.  The math is well-understood, one must merely chose
to use it.

             ***********************************************
The BEE-L mailing list is powered by L-Soft's renowned
LISTSERV(R) list management software.  For more information, go to:
http://www.lsoft.com/LISTSERV-powered.html