ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************

I think we also need to consider the societal context of these critiques 
of social science. With overall federal  funding cuts there is lobbying 
from the physics, engineering, and medical areas which have more 
political clout to maintain funding in the areas that support their own 
economic interests. Science as a societal endeavor has always been a 
value-laden and political process. It isn't surprising that these types 
of critiques come out at this time when scarcity has been created in the 
economic system. The prestige hierarchy of university sciences is also 
well-identified phenomena and when conflated with corporate interests, 
it is not an inconsequential factor in producing these critiques.

On the other hand, rigor is a topic that continues to evolve. 
Particularly in all applied areas, the traditional criteria for rigor 
(including repeatability/replication) are being called into question 
across multiple areas of science. Engineering and drug research have 
some ongoing issues, both with investigator independence (sponsors 
influencing results) and the initial focus and design blinding 
researchers from identifying essential questions.  For example, random 
control designs may be considered highly rigorous, but the length of the 
studies (influenced both by patient need and commercial interests)  
prevent the identification of serious, sometimes even fatal, side-effects.

Michael Quinn Patton recently recommended that evaluators look at the 
work of Zelik and his collaborators for better criteria for rigor. Zelik 
(et. al) looked at several situations (e.g. the Challenger Incident) 
where  where decisions, based of evidence, failed and have recommended 
criteria for rigor appropriate to applied situation. I think we need to 
consider this framework for rigor in informal science education evaluation.

Patton, M.Q. (2011).The debate about randomized controls in evaluation: 
The gold standard question. [PowerPoint slides in PDF form]. Retrieved 
from 
http://um.dk/en/~/media/UM/Danish-site/Documents/Danida/Resultater/Eval/Patton_RCT_April_2011pdf.ashx 
<http://um.dk/en/%7E/media/UM/Danish-site/Documents/Danida/Resultater/Eval/Patton_RCT_April_2011pdf.ashx>. 


Zelik, D. J., Patterson, E. S., & Woods, D. D. (2010). Measuring 
attributes of rigor in information analysis. In E. S. Patterson & J. E. 
Miller (Eds.), Macrocognition metrics and scenarios: Design and 
evaluation for real-world teams. Aldershot, UK: Ashgate. (ISBN: 
978-0-7546-7578-5) 
http://csel.eng.ohio-state.edu/zelik/research/Rigor_files/ZelikPattersonWoods_MeasuringRigor_1.pdf.

Zelik, D., Patterson, E. S., & Woods, D. D. (2007, June). Understanding 
rigor in information analysis. Paper presented at the 8th International 
Conference on Naturalistic Decision Making, Pacific Grove, CA. (PDF) 
http://csel.eng.ohiotate.edu/zelik/research/Rigor_files/ZelikPattersonWoods_UnderstandingRigor_1.pdf. 


Zelik, D., Patterson, E. S., & Woods, D. D. (2007, June). Understanding 
rigor in information analysis. Paper presented at the 8th International 
Conference on Naturalistic Decision Making, Pacific Grove, CA. (Video) 
_http://csel.eng.ohio-state.edu/zelik/research/Presentations-NDM8.html. _

Zelik, D., Patterson, E. S., & Woods, D. D. Modeling rigor in 
information analysis: A metric for rigor. 
http://csel.eng.ohio-state.edu/zelik/research/Rigor_files/ZelikPattersonWoods_ModelingRigorPoster.pdf. 



Best regards,


Carey


Carey Tisdal
Tisdal Consulting
[log in to unmask]





On 6/25/2012 2:17 PM, Eric Siegel wrote:
> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
> Incorporated, a worldwide network of science museums and related institutions.
> *****************************************************************************
>
> It is also fascinating because all EHR programs require that we hire social scientists to evaluate our programs.  In informal science education (whatever its named now), we are being encouraged--if not quite required--to integrate research about learning into each of our projects.
>
> So, if the principal problem of social science is the lack of predictive value, that would call into question the value of these evaluations.
>
> I am assuming that the learning sciences are included among the social sciences?
>
>
> Eric Siegel
> Director and Chief Content Officer
> New York Hall of Science
> 47-01 111th Street
> Queens, NY 11368
> 718-595-9117.
> www.nysci.org
>
> esiegel at nyscience dot org
> The Works blog at theworks.nysci.org
>
> NY Hall of Science is on:
>
> Twitter: http://twitter.com/nysci
> Facebook: http://www.facebook.com/nysci
> YouTube: http://www.youtube.com/user/nyhallofscience
>
>
>
>
>
>
>
>
>
>
> On Jun 25, 2012, at 2:40 PM, Maria Mortati wrote:
>
>> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
>> Incorporated, a worldwide network of science museums and related institutions.
>> *****************************************************************************
>>
>>>> I think the question of what's really science is also an important one for science museums to consider.
>> Yup, in practice "the sciences" vs "social science" are very different.
>>
>> That said, I find it intriguing that social science wouldn't be considered a factor in any NSF endeavor when all science is a human and therefore implicitly social endeavor.
>>
>> .
>> Maria Mortati, not a scientist, social or otherwise
>> .
>> (415) 235-8994
>> [log in to unmask]
>> .
>> site: http://www.mortati.com
>> blog: http://www.mortati.com/blog
>> .
>>
>>
>
>
>
>
>
>
>



***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.

Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.

The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.

To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]