ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************
Just to add to this discussion, here are some ideas form the evaluation
perspective--
One reason we don't have a strong, coherent body of findings related to
informal learning experiences is that we have not made explicit connections
between the program theory (how it's supposed to work) and activities (what
we do to make it work) and student outcomes. Especially with grant-funded
programs, funders want that one specific program to be a model for others
to
use. Sometimes we write grants we don't take that seriously by asking for
and including the time, resources and types of expertise to make that
happen. In experimental design research studies the researcher designs the
"treatment" to test a hypothesis. In program development/evaluation,
that "treatment" is the program itself, and the evaluator is assessing
someone else's hypothesis. It isn't appropriate for evaluators to walk too
far across that line. But we can collaborate and help develop processes
that allow that. It's a joint effort and those of us in evaluation can help
by moving the question from "Did it work?" to "What made it work and to
what
extent?" I know IMLS has been doing logic modeling workshops, and that is
a step in the right direction. Another great resource for how to do this
is:
W. S. Kellogg Foundation. (2001). Logic model development guide, [PDF].
W.K.
Kellogg Foundation. http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
Moving towards some explicit and shared program development/evaluation
models, similar to the exhibit development models we have shared with each
other over the past 15 years, would move us far ahead in making informal
learning experiences more accessible to kids in schools. This would enable
us to affect public policy.
Here is how, I think, that might look were we to do it:
Build in a six month to eight months start-up BEFORE the program is offered
to students. This time would allow program developers to work with
evaluators producing specific logic models, conducting front-end
evaluation,
writing detailed curriculum, and testing activities. It would also allow
time to hire staff. No organization can afford to take these steps without
a
firm funding source, and we are all constantly caught in the timing of
funding and trying to get things organized. We understand the necessity of
this phase in exhibit development--we need it in program development, too.
Also include in that start-up some team-building and instructional design
training so that the whole team understands the benefits of documenting
program activities throughout the process. I understand that sometimes
program staff feel these activities are "taking away time from the kids."
But, not doing these activities may be taking away access to these types of
learning experiences for a whole lot of kids when we don't have findings
strongly connected to program theory and specific
activities.
Well, there's my dream world. :-) I think most of us work in informal
learning organizations because we see it as the place where the joy and
satisfaction of learning is valued as an integral part of any kind of
effective learning. But, we need to include explicit program
design and documentation efforts that allows to test our belief in
systematic ways--and be credible to public policy makes, funders, parents,
and all the other people who aren't there everyday in the situation.
Carey
Carey Tisdal
Tisdal Consulting--Museum Research and Evaluation
Email: [log in to unmask]
***********************************************************************
More information about the Informal Science Education Network and the
Association of Science-Technology Centers may be found at http://www.astc.org.
To remove your e-mail address from the ISEN-ASTC-L list, send the
message SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]
|
