ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************
Josh,
Thanks so much these are great comments, and capture a shared intent.
C
On Jan 5, 2011, at 5:10 PM, Josh Gutwill wrote:
> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
> Incorporated, a worldwide network of science museums and related institutions.
> *****************************************************************************
>
> Hi Folks,
>
> I'd like to respond to Charlie Carlson's very provocative post. If I understand his concerns correctly, it seems like his questions fall into two categories:
>
> - What is the value of (summative) evaluation? What is it's purpose and efficacy? Is it worth the cost?
> - Do museums have an important or lasting impact on people? Has summative evaluation offered any insights into this?
>
> These are very important questions, and I'm sure many other practitioners (and even evaluators) share them. I don't think I’m in a position to directly answer whether evaluation is worth the cost, because it's a question of an institution's values, but I can try to illuminate the benefits of evaluation.
>
> I believe that incorporating evaluation into museum development processes has several benefits, including:
> - fostering a culture of reflection in practitioners by asking practitioners to define their goals and think more clearly about the learning process they hope to engender
> - defining what we mean by "learning." For example, the NSF ISE Evaluation Framework, which lays out different kinds of learning processes, may help practitioners distinguish between, say, learning concepts and practicing scientific thinking skills. I believe there is value in making these kinds of distinctions, though I know there are concerns about narrowing our definitions of learning too much. Still, I’m hopeful that our definitions and methods will continue to improve, bringing us closer to capturing the real and important kinds of learning that happen in museums.
> - illuminating the actual impact we have as institutions. As some have noted, over the past 10-15 years, NSF ISE has required that all funded projects include a summative evaluation study. Though this may be problematic for some, it has created a fund of knowledge about what works and what doesn’t, and about the kinds of learning that happen in museums. As Dave Ucko mentioned, the National Research Council released a summary report in 2009 (Learning Science in Informal Environments) drawing heavily on that knowledge base. NSF has also created a system (OPMS) that will allow us as a field to track impacts over time and across projects, using the Evaluation Framework. Perhaps over time, you'll see the kind of large-scale evidence you're craving.
> - attracting supporters and donors who wish to see evidence of impact before contributing their resources to our shared goal of promoting learning in informal environments.
>
> This leads me to Charlie's second question: Do science museums have an impact? According to the NRC committee’s report, which refers to museums as designed settings, there is “abundant evidence that across all venues—everyday experiences, designed settings, and programs—individuals of all ages learn science.” (NRC, 2009, p. ES-2). The report goes on to describe the kinds of learning, to offer evidence for it, and to present case studies to illustrate it. Some of the work cited is based on research studies, but much of it is based on evaluation. Without evaluation in museums and other informal learning environments, we would have precious little evidence of our impact. Charlie asks for evidence and proof of learning impact – the LSIE report was meant to be a summary of exactly that.
>
> I hope this helps move the conversation forward.
>
> Best,
>
> Josh
>
>
> --------
> Josh Gutwill, Ph.D.
> Visitor Research and Evaluation
> Exploratorium
> 3601 Lyon Street
> San Francisco, CA 94123
> 415-353-0408 (voice)
> 415-561-0370 (fax)
>
> ***********************************************************************
> For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.
>
> Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.
>
> The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
> http://www.lsoft.com/LISTSERV-powered.html.
>
> To remove your e-mail address from the ISEN-ASTC-L list, send the
> message SIGNOFF ISEN-ASTC-L in the BODY of a message to
> [log in to unmask]
Charles Carlson
Senior Scientist
exploratorium
3601 Lyon St.
San Francisco, CA 94123
[log in to unmask]
Tel: 415-561-0319
Fax: 415-561-0370
Skype: sciskypecharlie
MobileMe: [log in to unmask]
Twitter: charliec53
***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.
Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.
The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.
To remove your e-mail address from the ISEN-ASTC-L list, send the
message SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]
|
