"The current gold standard for certainly, a "P value" of 0.05 or less just does not mean what it used to, as researchers have been using "frequentist" statistical methods, which are "weak-kneed", there's also been some outlier data dropped on the floor more than once:"

My co-authors and I have started working with bootstrap statistics, we use a Monte Carlo program that my colleague designed.  It determines, using over >500,000 random sortings, what the likelihood is that our results were a result of chance and not the experimental design.  That method is highly conservative and we feel better about it than we do about using the "gold standard" of one of the usual ANOVA tests.  With this technique, data sets that *appear* significantly different when graphed (S.E. of the means don't overlap) are sometimes rejected by the Monte Carlo program.  It keeps us on the straight and narrow, in my opinion.

Statistics always need to be properly applied as in some instances, the statistical metric should test a Type 1 hypothesis, and in some, a Type 2...that is, in some studies the conservative approach is to try to disprove the hypothesis, and in some the conservative approach would be to prove the hypothesis. Unfortunately not all researchers can tell the difference. I always get help with this from a professional, as I'm not a statistician.  When my results aren't cut-and-dried obvious, I don't like to rely on my own weak math to figure out what they mean.

There was a big article about peer review in Science magazine recently.  I wish I could share it to the list, but all I can do is give you a brief synopsis and the citation.

Open access journals that anyone can get to have serious flaws where peer review is concerned.  I have served as a reviewer for some of these desktop open-access publishers and learned that when the authors fork over money to the publisher, little or no scientific oversight is in place or reinforced.  This means that when I review such a paper, I am usually the ONLY reviewer, and also I cannot produce an effective review because the responses have to fit into a "check-the-appropriate-box" format instead of a few paragraphs of prose.  Worse, there is no follow-up...that means the authors don't have to re-submit for follow up review after they "address" concerns by the reviewers with the paper.  The paper just gets published without further oversight. What that means is that you can publish literally any trash in an open access journal as long as you are willing to pay for it.

A Science editor submitted a bogus paper with completely fallacious methods to a large number of open access journals (he used a clever trick to be able to say they were all different...he used different species in each one...but the basic science was the same)  and had it accepted without question in more than 60% of the journals he submitted to.

Here is the reference:

Who's afraid of peer review?
Science 4 October 2013:
Vol. 342 no. 6154 pp. 60-65
DOI: 10.1126/science.342.6154.60

Those of us "in the trade" know which journals are well-reviewed, and we also know enough to decide for ourselves how much impact a paper has based on the work that was done, at least within our own field of study.  Students and laypeople usually can't do that.  This makes communicating science so much more difficult!!!!

Christina





             ***********************************************
The BEE-L mailing list is powered by L-Soft's renowned
LISTSERV(R) list management software.  For more information, go to:
http://www.lsoft.com/LISTSERV-powered.html