ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************

Charlie

Great questions especially for a semester or longer seminar but I think that
on a list like this the essence of your question will get lost.

I think it might be much more useful for you to site a published evaluation
study or two that you can ask pointed questions about that highlight your
concerns. Then I think members of the list can focus their responses and
lead to a more focused and useful discussion, as I know you would like.

Best for the New Year,

Martin

On Mon, Jan 3, 2011 at 5:01 AM, Charlie Carlson
<[log in to unmask]>wrote:

> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
> Incorporated, a worldwide network of science museums and related
> institutions.
>
> *****************************************************************************
>
> Hi All,
>
> For some years I've wondered about the efficacy of exhibit
> evaluation,wondered whether or not it is useful, or more directly a
> bureaucratic hurdle that provides useless and specious validation that
> satisfies an inner need and social, political need to feel affective.  A CYA
> exercise by politicians,bureaucrats, and museum professionals.
>
> To put it bluntly: Are museums and taxpayers spending a significant amount
> of money on something of questionable value?  Science Centers have been
> engaged in doing "rigorous evaluation" for more than 20 years, what do we
> have to show?  Personally, I can think of little direct evidence to support
> the continued emphasis on exhibit evaluation.  This is not out of malice
> towards evaluators, the human mind, or a field of inquiry, but more directed
> towards the usefulness of the evaluations generally.
>
> I'm sure this raised hackles, but let me proceed. It's not personal.  I
> firmly believe we're all well intended. I like most evaluators and value
> their opinions.  It is a complex field.  To cut to chase:
>
> Museum visits are, indeed, events, fraught with every personal and social
> dimension. As such they are part of noise and chatter of day to day
> existence. Importantly, museum visits are also brief––ever so brief, hours
> out of a year ( some fraction of 6570 waking hours annually).  Against this
> small fraction of useful, engaged hours,  what is the value of evaluation
> versus return on expense?  At best it's an infinitesimally small return.
>  This is typically born out by evaluation studies.
>
> On the face of it, anyone funding an exhibit needs to know that they're
> getting value for their commitment of resources, and more broadly whether or
> not it is having an intended effect.  These are important questions.
>  Personally, I'd love to change the world.  But let me begin by posing
> series of questions and likely answers:
>
> What are the key concepts that characterize an excellent exhibit or museum?
>  There's not much of a predictable profile here.  Many unevaluated,
> free-lanced exhibits have proven popular over the years.  The Exploratorium,
> prior to its evaluative stance for instance, Phenomena, Who Done It?, or the
> City Museum in St. Louis  are prime examples.  There is little consistent
> statistical evidence that argues for evaluation as an essential component of
> exhibit presentation. Everything has been evaluated but what are we
> comparing it against? Highly evaluated failures are more the norm, just
> about every modern exhibition has it's strengths, not many are popular.
> Evaluation has been imposed secondarily but can it be considered an
> essential component?  At best, it's a feel good exercise but's little more
> than an academic exercise.
>
> Is there evidence that evaluation has improved or positively modified an
> exhibit or exhibition?  I think the evidence is scant.  There needs to be a
> series of controlled double blind studies to validate such claims.  I don't
> know of any significant evidence but maybe I'm ignorant.
>
> How much do people generally remember of a museum visit?  The evidence
> suggests: very little specifically but there are many affective impacts;
> visitors are mostly either positive or negative towards the subjects
> exhibited.  Giant heart, coal mine, submarine, tactile gallery, the
> psychology show, etc. come to mind.
>
> Do the specifics of an exhibition make a difference in human behavior?
>  Probably not for most people.
>
> Is scientific accuracy important? Yes, but a mistake is okay (that's part
> of science too!)  And the wrong stuff won't stand any way.  Accuracy
> strongly reflects on the presenting institution more than the base of
> knowledge (who do you trust, anyway?).
>
> Has a museum exhibit changed the course of human history?  Probably not!
>  We've gone though several purported iterations of museum impact (are we on
> exhibit iteration 4,5,6 or 7?), and there is scant evidence of social
> impact.  The TB exhibition at the turn of the 20th century may be an
> exception to the more generalizable impacts, but it was medical with many
> health impacts. Mostly, it's all hubris directed towards continued funding.
>  It's an unfortunate side track.
>
> Is formative evaluation valuable? Yes, but is formal formative evaluation
> essential ? Maybe, exhibit developers may be just as skilled.  This is
> situationally dependent.
>
> Does a visit to a museum change the way people learn? Maybe a very, very
> little bit.  Probably so small as to be undetectable.  But a small change
> does make a difference.
>
> Museums are an expression of liberty, social freedom and wealth and as such
> manifest and amplify the values of  liberal free societies.
>
> Given my brief track here, I love to hear arguments contrary or pro.
>
> I've invested a lifetime working on this stuff.
>
> All the best for new year,
> C
>
>
>
>
>
>
> Charles Carlson
> Senior Scientist
> exploratorium
> 3601 Lyon St.
> San Francisco, CA 94123
> [log in to unmask]
> Tel:   415-561-0319
> Fax:  415-561-0370
>
> Skype: sciskypecharlie
> MobileMe: [log in to unmask]
> Twitter: charliec53
>
>
>
>
>
>
>
>
>
>
>
> ***********************************************************************
> For information about the Association of Science-Technology Centers and the
> Informal Science Education Network please visit www.astc.org.
>
> Check out the latest case studies and reviews on ExhibitFiles at
> www.exhibitfiles.org.
>
> The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To
> learn more, visit
> http://www.lsoft.com/LISTSERV-powered.html.
>
> To remove your e-mail address from the ISEN-ASTC-L list, send the
> message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
> [log in to unmask]
>



-- 
–––––––––––––––––––––––––––––––––––––––––
Martin Weiss, PhD
Science Interpretation, Consultant
New York Hall of Science

***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.

Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.

The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.

To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]