ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************

So what's this idea of "star" as the criteria for something being valuable?
What's the criteria for being "a star"? Turning a "flop" into a "star" is
not the definition of the role of evaluation I use. Evaluation should not be
confused with alchemy. :-) The role of formative exhibit evaluation isn't to
change a "flop" into "a star".  Whatever your critera may be, the role of
formative evaluation is to support the improvement of an entity before it
goes out to the public. It's good to have some agreed on criteria about what
improvement means before you start that process--the funders,
administrators, designers, and visitors may all have different critera.

Sometimes, the skepticism about evaluation is NOT open to emprical evidence.
For example, there have been discussions about evaluation here before--Ed
Rodley and Josh Gutwill (to name two responders I remember) and both had
well-reasoned replies with many examples about the of value.  Rather, I 
think
that many times the skepticism is a question of identity and 
self-perception. And you know what?
Sometimes people who have that self-perceptions as the lone, untamed
creative genius create some experiences that are just wonderful to me as a
visitor--I live in St. Louis and I love the City Museum. Bob Casilly did not
like the collaborative process; he was the lone creative type. But there are
some things at the City Museum I won't do. (And some areas that aren't 
wonderful, just cluttered and choatic.) There have been too many people
hurt and too many lawsuits. I don't trust that the appearance of risk is
simply the appearance of risk at the City Museum. I am not risk averse. (My
career choices and my driving should prove that!). But, Bob Cassilly's loved
physical risk and it is a theme that ran through his life, work, and death. 
And risk and pushing against the norms is not the only type of good 
experience.

Done well, formation evaluation can also become the safety net that let's
people take great risks and let the bad things on a small scale before
 a lot people get to have a bad experience. And what about the many
efforts that were simply more meaningful because after prototyping, visitors
could use the interface? What about labels that "made sense" to people after
some testing and revision? What about themes that connected because there
were focus groups that allowed designers to see some blank and confused
faces looking at some of conceptual schemes and some interest and excitement
looking at others? This idea that the only thing that is valuable is the
great iconic exhibiton is just that--one thing that is valuable. There are
numerous examples of things that are better, clearer, more enjoyable.

I really do think a great example I can think of where evaluation made a
real difference is the magnetic levitated train engineering design lab in
Star Wars. The designers and internal evaluation unit at MOS prototyped,
prototyped, and prototyped. It wasn't just a brilliant idea (though as an
idea it was very cool), but it was also hard work--so interation after
interation was produced and tested to get people engaged enough to work
through three stations and actually make a train that worked.

Frankly, in many instances where formative evaluation did make big
differences, you may never know about it. Evaluation is part of the process
and it often needs to be done somewhat seamlessly and quietly to make a 
difference, to
focus on the things that do work, the ideas that do have promise, and help a
team leave those that don't at the story-boarding stage or on the shop
floor without shaking the confidence and shutting down the process coming up 
with the ideas. My friend and mentor Kathryn Nelson who worked on many 
desegregation efforts said, "You can facilitate change or you can take 
credit for change, but you can't do both."

If you are making the point that some (not all!) exhibit and exhibition 
development has
become too focused on the didactic, with a focus on short-term cognitive
outcomes rather than pleasure, beauty, fun, satisfaction, humor,
questionning accepted ideas, and so forth, I agree with you totally. But
that is not evaluation, that is a larger cultural movement where the
underlying assumptions of positivistic science have been enacted in all
sorts fields.

Too much time on ISEN last night and this morning. I'm out of here. Have a 
good week-end Charlie! May all your evaluation experiences be rich, 
productive, thought provoking, and valuable.

Carey

Carey Tisdal
Tisdal Consulting
[log in to unmask]
314-496-9097

***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.

Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.

The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.

To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]