ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************

Jonah,

There are many good evaluation designs that you can draw from in 
evaluating these types of after school program. 

One of the key components of getting started, however, is to 
systematically define your goals and outcomes from the beginning so that 
you then choose the appropriate indicators and evaluation design to 
assess those specific outcomes.  You should consider developing a logic 
model for these programs to help in the process.   Otherwise, deciding 
on indicators of success and data collection methods first is a bit like 
putting the cart before the horse.

Also, have you considered working with a local professional evaluator to 
help you develop and evaluate the program?  (There are many in the 
northeast!) Universities are sometimes also good resources (look to 
education, psychology, or anthropology programs).  

You can also contact me off line and I can send some good additional 
publications and resources that might be helpful.

Good luck!

Cecilia


Jonah Cohen wrote:

>ISEN-ASTC-L is a service of the Association of Science-Technology Centers
>Incorporated, a worldwide network of science museums and related institutions.
>*****************************************************************************
>
>OK, I ask you, my cohorts for advice, and please tell me if my gut
>reaction is way off.
>
> 
>
>Here's the deal: we've gotten mucho $$$ to do after school programs at a
>number of local Boys & Girls Clubs. We'll be doing ~10 programs at each
>club. So far so good. Apparently, one of the stipulations of the grant
>is that we have to assess the impact our programs have on the kids at
>B&G. Our bigwigs have tentatively decided we should come up with a bunch
>of vocabulary words, give the kids a quiz before the programs, then give
>them one after the quiz to see if they've learned anything.
>
> 
>
>My gut reaction? This is complete {#$&*-Censored- (!^`#$} Here's why:
>
> 
>
>1)       It takes our programs, designed to cover a wide spectrum of
>scientific knowledge, and reduces it to rote regurgitation of a couple
>words. Yech. Not that vocabulary isn't important or anything, but if,
>say, one of our live animal programs demonstrates how animals have to be
>well suited to live in their homes (for example, we show how a ferret's
>flexible body is useful for maneuvering underground) kids can grok that
>even if they don't recall the meaning of "adaptation" or "subterranean".
>I'd prefer that they do, but assuming that the ability to do so is the
>best way to assess our stuff doesn't make sense to me.
>
>2)       Oy, what a logistical nightmare. We've done programs at Boys &
>Girls Clubs before. It's pretty chaotic. Every week the kids who show up
>are different. Often their parents are picking them up mid-program. How
>on earth are we supposed to assess the kids based on who attended which
>program and was there for specific vocab or other info. Or for that
>matter, who shows up for the assessment.
>
>3)       Worst of all, a Boys & Girls Club isn't school. Kids go there
>for fun, not to be tested (again). A big part of what we do involves
>getting kids to have a positive attitude about science, to enjoy
>science, to get confidence that they can do science. If they see it as
>(yet another) round of testing, they may simply not show up for our
>programs and just go to B&G to play basketball or other things.
>
> 
>
>I'm sure these beefs sound familiar, as the listserv has seen much talk
>about standardized tests - but the fact that they're now being
>transferred to non-school environments is an annoying wrinkle. So, now
>that I've vented/made my case, my questions to you are:
>
> 
>
>1)       Am I nuts in my above complaints?
>
>2)       Anyone have any experience assessing the impact of programs
>like this in a setting as thoroughly informal as B&G? Any advice? Help
>me. Please.
>
> 
>
>Jonah Cohen
>
>Outreach & Public Programs Manager
>
>The Children's Museum (formerly The Science Center of Connecticut)
>
> 
>
>"Connecticut is called The Nutmeg State, because much like the nutmeg in
>any recipe it's ever been in, no one would miss it if it were gone."
>
>         -Jon Stewart
>
> 
>
>
>***********************************************************************
>More information about the Informal Science Education Network and the
>Association of Science-Technology Centers may be found at http://www.astc.org.
>To remove your e-mail address from the ISEN-ASTC-L list, send the
>message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
>[log in to unmask]
>
>
>  
>


-- 

_____________________________________

 

Cecilia Garibay

Principal

Garibay Group

phone: 773-271-5843

fax: 425-790-6317

email: [log in to unmask] <mailto:[log in to unmask]>

______________________________________

 


***********************************************************************
More information about the Informal Science Education Network and the
Association of Science-Technology Centers may be found at http://www.astc.org.
To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]