ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************

Hi All Respondents to date,

I've very much enjoyed much of the discussion so far and want to thank everyone for thoughtful and considered opinion, evidence and argument.  Clearly, in spite of my best efforts things get a little personal, and I was and am provocative at times.  I think it's a tricky and contentious area.  The arguments, opinions, errors  presented in my initial arguments are strictly mine, and come from my personal perspective, and are not meant with malice towards the field.  A few nerves have been touched. 

I propose to make a summary of the relevant comments I've received by the end of this week or next, and then I'd like to tease them out point by point.  My intention is to foster better communication as well as answer questions I've long held.  I agree with those that want more specific references.  I'll do my best.  

Amongst the many comments and replies received to date, I found the comparisons between a visit to an art museum and its effects poignantly interesting.  Dave Ucko's comments provide lots of insight, as do Friedman's and others.  Thanks!  I think it's a really great discussion, and reflects 40 plus years of effort collectively by many, many dedicated people.

Ultimately, I would like to see a clearer division between research and exhibition metrics.  I don't know if this possible.  I would also like to see a clearer theory of informal learning that might be tested.  I don't know if that is possible either.  It seems to me that there is so much wonderful new research about the brain, and cognitive behavior that folds into the picture.  I do reflect on Judy Diamond's first tracking and timing studies at the Exploratorium in the mid-seventies; it was an eye-opening experience for me, as was the work of Minda Borun and others.

Sincerely,
Charlie
On Jan 3, 2011, at 2:01 AM, Charlie Carlson wrote:

> Hi All, 
> 
> For some years I've wondered about the efficacy of exhibit evaluation,wondered whether or not it is useful, or more directly a bureaucratic hurdle that provides useless and specious validation that satisfies an inner need and social, political need to feel affective.  A CYA exercise by politicians,bureaucrats, and museum professionals.
> 
> To put it bluntly: Are museums and taxpayers spending a significant amount of money on something of questionable value?  Science Centers have been engaged in doing "rigorous evaluation" for more than 20 years, what do we have to show?  Personally, I can think of little direct evidence to support the continued emphasis on exhibit evaluation.  This is not out of malice towards evaluators, the human mind, or a field of inquiry, but more directed towards the usefulness of the evaluations generally. 
> 
> I'm sure this raised hackles, but let me proceed. It's not personal.  I firmly believe we're all well intended. I like most evaluators and value their opinions.  It is a complex field.  To cut to chase:
> 
> Museum visits are, indeed, events, fraught with every personal and social dimension. As such they are part of noise and chatter of day to day existence. Importantly, museum visits are also brief––ever so brief, hours out of a year ( some fraction of 6570 waking hours annually).  Against this small fraction of useful, engaged hours,  what is the value of evaluation versus return on expense?  At best it's an infinitesimally small return.  This is typically born out by evaluation studies.
> 
> On the face of it, anyone funding an exhibit needs to know that they're getting value for their commitment of resources, and more broadly whether or not it is having an intended effect.  These are important questions.  Personally, I'd love to change the world.  But let me begin by posing series of questions and likely answers:
> 
> What are the key concepts that characterize an excellent exhibit or museum?  There's not much of a predictable profile here.  Many unevaluated, free-lanced exhibits have proven popular over the years.  The Exploratorium, prior to its evaluative stance for instance, Phenomena, Who Done It?, or the City Museum in St. Louis  are prime examples.  There is little consistent statistical evidence that argues for evaluation as an essential component of exhibit presentation. Everything has been evaluated but what are we comparing it against? Highly evaluated failures are more the norm, just about every modern exhibition has it's strengths, not many are popular.   Evaluation has been imposed secondarily but can it be considered an essential component?  At best, it's a feel good exercise but's little more than an academic exercise.
> 
> Is there evidence that evaluation has improved or positively modified an exhibit or exhibition?  I think the evidence is scant.  There needs to be a series of controlled double blind studies to validate such claims.  I don't know of any significant evidence but maybe I'm ignorant.
> 
> How much do people generally remember of a museum visit?  The evidence suggests: very little specifically but there are many affective impacts; visitors are mostly either positive or negative towards the subjects exhibited.  Giant heart, coal mine, submarine, tactile gallery, the psychology show, etc. come to mind.
> 
> Do the specifics of an exhibition make a difference in human behavior?  Probably not for most people.
> 
> Is scientific accuracy important? Yes, but a mistake is okay (that's part of science too!)  And the wrong stuff won't stand any way.  Accuracy strongly reflects on the presenting institution more than the base of knowledge (who do you trust, anyway?).
> 
> Has a museum exhibit changed the course of human history?  Probably not!  We've gone though several purported iterations of museum impact (are we on exhibit iteration 4,5,6 or 7?), and there is scant evidence of social impact.  The TB exhibition at the turn of the 20th century may be an exception to the more generalizable impacts, but it was medical with many health impacts. Mostly, it's all hubris directed towards continued funding.  It's an unfortunate side track.
> 
> Is formative evaluation valuable? Yes, but is formal formative evaluation essential ? Maybe, exhibit developers may be just as skilled.  This is situationally dependent.
> 
> Does a visit to a museum change the way people learn? Maybe a very, very little bit.  Probably so small as to be undetectable.  But a small change does make a difference.
> 
> Museums are an expression of liberty, social freedom and wealth and as such manifest and amplify the values of  liberal free societies.
> 
> Given my brief track here, I love to hear arguments contrary or pro. 
> 
> I've invested a lifetime working on this stuff.
> 
> All the best for new year,
> C
> 
> 
> 
> 
> 
> 
> Charles Carlson
> Senior Scientist
> exploratorium
> 3601 Lyon St.
> San Francisco, CA 94123
> [log in to unmask]
> Tel:   415-561-0319
> Fax:  415-561-0370
> 
> Skype: sciskypecharlie
> MobileMe: [log in to unmask]
> Twitter: charliec53
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Charles Carlson
Senior Scientist
exploratorium
3601 Lyon St.
San Francisco, CA 94123
[log in to unmask]
Tel:   415-561-0319
Fax:  415-561-0370

Skype: sciskypecharlie
MobileMe: [log in to unmask]
Twitter: charliec53











***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.

Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.

The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.

To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]