ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************
Thanks for the very useful information, Rita.
Eric Siegel
Director of Planning
& Program Development
Connections Project Director
New York Hall of Science
47-01 111th Street
Queens, NY 11368
[log in to unmask]
www.nyscience.org
> -----Original Message-----
> From: Informal Science Education Network
> [mailto:[log in to unmask]] On Behalf Of Rita Deedrick
> Sent: Friday, May 28, 2004 10:14 AM
> To: [log in to unmask]
> Subject: Re: Evaluation Suggestions Wanted
>
>
> ISEN-ASTC-L is a service of the Association of
> Science-Technology Centers Incorporated, a worldwide network
> of science museums and related institutions.
> **************************************************************
> ***************
>
> I don't disagree with David Taylor's comment about the lack
> of evidence showing long-term cognitive change from
> short-term science center experiences. But, that doesn't
> mean you can't engage in meaningful evaluation. The first
> step is to be brutally honest in determining what it is that
> you want to measure. Often our program goals and aspirations
> are so lofty that it's downright frightening to try to
> measure real change. But we have to be honest with ourselves
> in what change we can really expect to happen as a result of
> a program and then design an evaluation accordingly.
>
> While setting realistic evaluation goals is the first step,
> the last step is just as important. That is to be careful
> how you interpret the data and how you state the findings. A
> recent evaluation of one of COSI's outreach programs showed
> statistically significant difference between how a sample of
> students answered "knowledge" and "attitude" questions on a
> post-program survey from their pre-program responses, but we
> are careful not to suggest a long-term impact. We simply
> don't have the resources to do a long-term follow up, and we
> won't say that the program achieved a long-term result. We
> can simply say that the students' knowledge and attitude
> improved from immediately before the program to immediately
> after the program because that's all we measured.
>
> A few practical notes about the two studies we've done on our
> outreach programs. First, we had a little funding to hire a
> professional evaluation consultant, and an outside eye is
> very helpful in taking that honest look at measurable goals.
> As I mentioned already, we used a pre/post test method in
> surveying students on a series of knowledge and attitude
> questions. We took advantage of the structure of the formal
> classroom to have the teachers administer the surveys for us.
> Two caveats here* One is that it is very difficult for
> students and teachers to see the surveys as something other
> than "tests for a grade" and there's a good deal of anxiety
> about the students not being able to answer the questions,
> especially on the pre-program test. It's hard for them to
> see this as something other than student assessment, so you
> need to do a lot of messaging about how these tests are
> different from school tests. Secondly, it took much more
> coaxing, reminding, and general handholding of the teachers
> than I ever expected to get this done. Teachers are so
> overwhelmed in today's classrooms that this simple study was
> burdensome to some. I had better luck when I got the support
> of the principal first, and used a lot of patience in
> developing a continuing rapport with the teachers. And
> remember, you don't want to keep going back to the same
> teachers that you already know well. A more representative
> sample will include teachers who are not necessarily
> "friends" of the science center.
>
> Good luck! 'Would be happy to discuss in more detail off-line.
>
> Rita Deedrick
> Senior Director for Program Evaluation
> COSI Columbus
>
> Return to the Days of the Dinosaur!
> Dinosaur Dynasty: Discoveries from China at COSI Columbus
> May 8 - September 6, 2004
>
> GroupWise powered by Time-Warner Road Runner
>
> >>> [log in to unmask] 05/27/04 09:07PM >>>
> ISEN-ASTC-L is a service of the Association of
> Science-Technology Centers Incorporated, a worldwide network
> of science museums and related institutions.
> **************************************************************
> ***************
>
> Aloha,
> I am trying to establish an evaluation procedure to assess
> the extent to which our outreach programs accomplish our
> stated goals, specifically in terms of student learning.
> Does anyone have any advice or recommendations about how to
> get a true picture of the impact our program had on students?
> We are interested in both what content students learned and
> how their attitudes changed because of our program. This is
> not intended to evaluate how well the students or teacher
> liked the program. I am mainly considering some type of
> pre/post test that teachers conduct and return to us, but if
> anyone has other ideas I'd be happy to hear them. Also, if
> anyone has evaluation samples they are willing to share, that
> would be great. Thanks, Nancy
>
> ****************
> Nancy Alima Ali
> Science Educator
> Bishop Museum
> 1525 Bernice Street
> Honolulu, HI 96817
> Phone:(808) 847-8227
> Fax: (808) 848-4155
>
> **************************************************************
> *********
> More information about the Informal Science Education Network
> and the Association of Science-Technology Centers may be
> found at http://www.astc.org. To remove your e-mail address
> from the ISEN-ASTC-L list, send the message SIGNOFF
> ISEN-ASTC-L in the BODY of a message to [log in to unmask]
>
> **************************************************************
> *********
> More information about the Informal Science Education Network
> and the Association of Science-Technology Centers may be
> found at http://www.astc.org. To remove your e-mail address
> from the ISEN-ASTC-L list, send the message SIGNOFF
> ISEN-ASTC-L in the BODY of a message to [log in to unmask]
>
***********************************************************************
More information about the Informal Science Education Network and the
Association of Science-Technology Centers may be found at http://www.astc.org.
To remove your e-mail address from the ISEN-ASTC-L list, send the
message SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]
|
