[log in to unmask] (mailto:[log in to unmask]) writes:
Only Maryann Frazier got through to ask a hard question. She made
reference to the sense that EPA is not giving equal value to academic research
because it does not follow some sort of standard of "best research practices".
The EPA guy stated, of course, it wasn't true, that they take everything
into account.
EPA mandates across all of its labs and contractors something called Good
Laboratory Practices or GLP. There's no secret, and its easy to do, but it
does take time and adds to costs.
Its a way of controlling quality. All experiments have to develop a
quality assurance plan, and steps need to be outlined and followed to
demonstrate the 'quality' of the work. Following written plans, developing and
following standard procedures, documenting the work, and undergoing independent
audits (in other words, EPA is likely to show up unannounced and conduct an
audit of our research logs - and the answer better not be ' log books, what
log books?) are routine parts of GLP, sometimes called QA/QC.
Under GLP, if you need to weigh out something, you need to use standard
weights to be sure that the scale is accurate and generally you need to use
a certified scale (just as the gas station has to certify the accuracy of
their pumps). If temperature is critical, you don't use the thermometer
from the discount rack at the hardware store unless you can calibrate it or
show that it is traceable to a calibrated thermometer. I have some NIST
(Nat Institute of Standards Traceable) thermometers. We don't use them for
day to day measurements, we use them whenever we get in a new thermometer or
use one that hasn't been tested recently. The daily use thermometer does
not have to be 100% accurate, It does have to yield reproducible results.
If I compare a cheap hardware store thermometer, it is likely to be
consistently off by a few degrees. If you can document the amount that its off
the accurate value, you have 'traced' it to NIST, and you can use it.
Also, even the NIST thermometer isn't on the money all across its range.
It comes with a calibration sheet showing the amount over or under that it
is at several points along the range - usually five or more.
If you dose bees with a pesticide, it is not sufficient to proclaim what
you calculate the dose to be, you have to verify the calculated dose by
chemical analysis. One can always make a mistake in calculations, in a
measurement, or the chemical may simply break down before it can be tested, or
even be mislabelled.
QA (quality assurance) QC (quality control) is common in many industries -
Henry Ford was one of the first to apply it to the manufacture of
automobiles.
None of this is unique to pesticide work and bees. It applies to more or
less everything that EPA does or gathers data for. I've followed GLP for
years in our work on EPA Hazardous Waste sites.
Many university labs follow QA/QC and GLP. Virtually all private labs
have to, or else, no one would use them. Quality Assurance is even a career
option - and some universities include it in their curricula.
GLP can't make up for errors such as poor experimental design, or lab or
field studies that don't really simulate what happens with real colonies of
bees in the field. It can reduce experimenter error, and it provides a
measure of how good the measurements are.
For example, if one analyzes a bee sample for pesticides, no instrument is
going to 'recover' the exact amount in the sample. Depending on the
instrument, experience of the technician, the sample matrix, and even the
weather, the instrument may over or under-estimate the real value. That's not
unexpected nor necessarily a bad thing. QA/QC allows one to flag results
that are out of compliance or out of control AND for accepted results, it
provides a means of assessing accuracy and precision.
With complex organic chemicals, 100% recovery is uncommon. Don't be
surprised to see something like 102% recovery. More often, something less than
100% is typical. Send samples to two different labs and you will likely to
see a lab bias, where one lab is consistently high or low compared to the
other.
Over the course of my career, I've done thousands of analyses for a wide
array of environmental chemicals including pesticides and pollutants. If
I'm analyzing for copper in bee tissue using ICP, I expect to get a result
that is within + 1 to 5 ppm of the real value. Its a simple matrix, and
copper is an easy to detect chemical, and it occurs at high levels in the
envionment.
When analyzigng multiple complex organic chemicals at a few ppb level, an
analyst might be happy with a result in the range of 85-105% of the real
value. In fact, complex organic mixtures are often analyzed in triplicate
(three times over) because the recoveries are so variable. The reported
result is often the geometric average of the three, adjusted from instrument
bias.
There are EPA guidelines for analysts to follow concerning how to
determine when an analytical result is out of compliance. The official terms are -
does the sample exceed warning limits?, does it exceed control limits?
The first requires checking and maybe adjusting the instrument; the second
requires re-running of all sample analyzed since the last time the
recoveries were at acceptable levels.
For EPA GLP, this typically means that every 10 - 15 samples, the analyst
has to re-run a calibration check. If everything is ok, the value is
recorded, and the analysis proceeds. If it NOT, then one has to back track and
re-run all samples analyzed since the last time the instrument could be
shown to be in compliance.
THIS is not an unreasonable requirement. You don't expect the concrete
contractor to pour your foundation and not have to meet specific engineering
and local code requirements.
The food you eat in a restaurant is checked by health inspectors who check
temps in freezers, warming plates, sample for E. Coli, etc.
The air pressure in your tires (if you have a modern vehicle) is monitored
to reduce chance of blow outs on the highway.
If one is doing basic 'academic' research on something that is exploratory
or that no one really cares about except that it provides a new idea, one
may be able to ease up a bit on the formal GLP (QA/QP), Even then, good
science requires meticulous and reproducible results. GLP should be the
norm, not something to be avoided.
Years ago, I delivered a talk to my colleagues in the Division of Biology.
One of our avian specialist got upset, proclaimed QA/QC a waste of time.
I asked him if he conducted annual bird surveys with student volunteers.
He answered in the affirmative.
I then told him that if he and I were to walk through a wood lot, I knew
he'd see and hear and correctly ID more birds than I - and he agreed. I
then asked how he assessed how good each of the students was at seeing,
hearing, and IDing birds. He said he trained them.
I asked how he knew whether they learned their lessons. Did any have
hearing or vision impairments? How experienced were they?
Did he get some mounted birds or make up some artificial birds and hide
them in the woods? Did he place audio systems in the bushes and playback
bird songs? Did he test to see if a difference in this year's bird count
was actually due to a difference in the diversity and abundance of birds or
was it due to a group of bird watchers who were either better or worse than
average?
He walked off in a huff. A few days later, he came to me and apologized.
He realized he was simply being defensive because he'd never thought
about this. He then started working on improving the bird counts and went on
to publish papers on sources of error and how to improve the accuracy of
annual surveys.
EPA has every reason to ask for something similar from experimental work,
where the outcome has very real ecological, economic, or health
consequences. Critical questions requires rigorous methods to provide good
information to decision makers.
From my perspective, all university level research should be conducted
following appropriate QA/QC (GLP). Its part of the training that we have a
professional and ethical responsibility to provide to our students.
Jerry
***********************************************
The BEE-L mailing list is powered by L-Soft's renowned
LISTSERV(R) list management software. For more information, go to:
http://www.lsoft.com/LISTSERV-powered.html
|