[log in to unmask] (mailto:[log in to unmask])  writes:

Only  Maryann Frazier got through to ask a hard question. She made 
reference to the  sense that EPA is not giving equal value to academic research 
because it does  not follow some sort of standard of "best research practices". 
The EPA guy  stated, of course, it wasn't true, that they take everything 
into account.  
EPA mandates across all of its labs and contractors something  called Good 
Laboratory Practices or GLP.  There's no secret, and its easy  to do, but it 
does take time and adds to costs.  
 
Its a way of controlling quality.  All experiments have to  develop a 
quality assurance plan, and steps need to be outlined and followed to  
demonstrate the 'quality' of the work. Following written plans, developing  and 
following standard procedures, documenting the work, and  undergoing independent 
audits (in other words, EPA is likely to show up  unannounced and conduct an 
audit of our research logs - and the answer better  not be ' log books, what 
log books?) are routine parts of GLP, sometimes called  QA/QC.
 
Under GLP,  if you  need to weigh out something, you  need to use standard 
weights to be sure that the scale is accurate and generally  you need to use 
a certified scale (just as the gas station has to certify  the accuracy of 
their pumps).  If temperature is critical, you  don't  use the thermometer 
from the discount rack at the hardware store unless you can  calibrate it or 
show that it is traceable to a calibrated thermometer.  I  have some NIST 
(Nat Institute of Standards Traceable) thermometers.  We  don't use them for 
day to day measurements, we use them whenever we get in a new  thermometer or 
use one that hasn't been tested recently.  The daily use  thermometer does 
not have to be 100% accurate,  It does have to yield  reproducible results. 
If I compare a cheap hardware store thermometer, it is  likely to be 
consistently off by a few degrees.  If you  can document  the amount that its off 
the accurate value, you have 'traced' it to NIST, and  you can use it.
 
Also, even the NIST thermometer isn't on the money all  across its  range.  
It comes with a calibration sheet showing the amount over or under  that it 
is at several points along the range - usually five or more.
 
 
If you dose bees with a pesticide, it is not sufficient to proclaim  what 
you calculate the dose to be, you  have to verify the calculated dose  by 
chemical analysis.  One can always make a mistake in calculations, in a  
measurement, or the chemical may simply break down before it can be tested, or  
even be mislabelled.  
 
QA (quality assurance) QC (quality control) is common in many  industries - 
Henry Ford was one of the first to apply it to the manufacture of  
automobiles.
 
None of this  is unique to pesticide work and bees.   It applies to more or 
less everything that EPA does or gathers data for.   I've followed GLP for 
years in our work on EPA Hazardous Waste  sites.
 
Many university labs follow QA/QC and GLP.  Virtually all  private labs 
have to, or else, no one would use them.  Quality Assurance is  even a career 
option - and some universities include it in their  curricula.
 
GLP can't make up for errors such as poor experimental design, or lab or  
field studies that don't really simulate what happens with real colonies of 
bees  in the field.  It can reduce experimenter error, and it provides a 
measure  of how good the measurements are.
 
For example, if one analyzes a bee sample for pesticides, no instrument is  
going to 'recover' the exact amount in the sample.  Depending on the  
instrument, experience of the technician, the sample matrix, and even the  
weather, the instrument may over or under-estimate the real value.   That's not 
unexpected nor necessarily a bad thing.  QA/QC allows one to  flag results 
that are out of compliance or out of control AND for accepted  results, it 
provides a means of assessing accuracy and precision.
 
With complex organic chemicals, 100% recovery is uncommon.  Don't be  
surprised to see something like 102% recovery.  More often, something less  than 
100% is typical.  Send samples to two different labs and you will  likely to 
see a lab bias, where one lab is consistently high or low compared to  the 
other.
  
Over the course of my career, I've done thousands of analyses for a wide  
array of environmental chemicals including pesticides and pollutants.  If  
I'm analyzing for copper in bee tissue using ICP, I expect to get a result 
that  is within  + 1 to 5 ppm of the real value.    Its a  simple matrix, and  
copper is an easy to detect chemical, and it occurs at  high levels in the 
envionment.
 
When analyzigng multiple complex organic chemicals at a  few ppb level, an 
analyst might be happy with a result in the range of  85-105% of the real 
value.  In fact, complex organic mixtures are  often analyzed in triplicate 
(three times over) because the recoveries are so  variable.  The reported 
result is often the geometric average of the  three, adjusted from instrument 
bias.
 
There are EPA guidelines for analysts to follow concerning how to  
determine when an analytical result is out of compliance.  The official  terms are - 
does the sample exceed warning limits?, does it exceed control  limits?
 
The first requires checking and maybe adjusting the instrument; the second  
requires re-running of all sample analyzed since the last time the 
recoveries  were at acceptable levels.  
 
For EPA GLP, this typically means that every 10 - 15 samples, the  analyst 
has to re-run a calibration check.  If everything is ok, the  value is 
recorded, and the analysis proceeds.  If it NOT, then one has to  back track and  
re-run all samples analyzed since the last time the  instrument could be 
shown to be in compliance.
 
THIS is not an unreasonable requirement.  You don't expect the  concrete 
contractor to pour your foundation and not have to meet specific  engineering  
and local code requirements. 
 
The food you eat in a restaurant is checked by health inspectors who  check 
temps in freezers, warming plates, sample for E. Coli, etc.
 
The air pressure in your tires (if you have a modern vehicle) is monitored  
to reduce chance of blow outs on the highway.
 
If one is doing basic 'academic' research on something that is  exploratory 
or that no one really cares about except that it provides a new  idea, one 
may  be able to ease up a bit on the formal GLP  (QA/QP),    Even then, good 
science requires meticulous and  reproducible results.  GLP should be the 
norm, not something to be  avoided.
 
Years ago, I delivered a talk to my colleagues in the Division of  Biology. 
 One of our avian specialist got upset, proclaimed QA/QC a waste  of time.  
I asked him if he conducted annual bird surveys with student  volunteers.  
He answered in the affirmative. 
 
I then told him that if he and I were to walk through a wood lot, I knew  
he'd see and hear and correctly ID more birds than I - and he agreed.   I 
then asked how he assessed how good each of the students was at seeing,  
hearing, and IDing birds.  He said he trained them.  
 
I asked how he knew whether they learned their lessons.  Did any have  
hearing or vision impairments?  How experienced were they?  
 
Did he get some mounted birds or make up  some artificial birds and  hide 
them in the woods?  Did he  place audio systems in the  bushes and playback 
bird songs?  Did he  test to see if a  difference in this year's bird count 
was actually due to a difference in  the diversity and abundance of birds or 
was it due to a group of bird  watchers who were either better or worse than 
average?
 
He walked off in a huff.  A few days later, he  came to me and  apologized. 
 He realized he was simply being defensive because he'd never  thought 
about this.  He then started working on improving the bird counts  and went on 
to publish papers on sources of error and how to improve the  accuracy of 
annual surveys.
 
EPA has every reason to ask for something similar from experimental work,  
where the outcome has very real ecological, economic, or health  
consequences.  Critical questions requires rigorous methods to provide good  
information to decision  makers.
 
From my perspective, all university level research should be conducted  
following appropriate QA/QC (GLP).  Its part of the training that we have a  
professional and ethical responsibility to provide to our students.
 
Jerry
 
 
 
 
 
 
 


             ***********************************************
The BEE-L mailing list is powered by L-Soft's renowned
LISTSERV(R) list management software.  For more information, go to:
http://www.lsoft.com/LISTSERV-powered.html