ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************

Hi,

For me, paragraphs 2 and 3 are a great synopsis of people and museums. Also, I really like the trajectory around rigor. I'm a process nerd and I tend to think anytime you're honing your craft it's a good thing.

> Museums directly and broadly increase public accessibility to our understanding of aspects of the natural world and allow us to explore it.

I would like to suggest that what all museums primarily excel at are isolating ideas and providing deep knowledge about them. Whether through exhibits or events.
.
Maria Mortati
.
(415) 235-8994
[log in to unmask]
.
site: http://www.mortati.com
blog: http://www.mortati.com/blog
.

On Jun 28, 2012, at 8:27 AM, Charlie Carlson wrote:

> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
> Incorporated, a worldwide network of science museums and related institutions.
> *****************************************************************************
> 
> Hi All,
> 
> The fundamental question I ask is not about the value of informal learning and our desire to better understand it, nor do I criticize the need for rigor in research or the appropriate use of randomized controlled trials.  I support all that.  I direct my criticism towards the basic question of whether or not museums are useful or appropriate venues in which to conduct such research.  It is not about the value of museums.
> 
> The observation to be made about any research model is to find one that's not too simple, has interesting behaviors, but is not so complex as to be difficult to analyze.  Thinking about it: For all the visitor research that's been conducted over the past 30 years what have we learned? We've learned that people enjoy museums. That they enjoy social experiences for a variety of reasons. That they like working with things and discovering things about the natural world.  That they use their hands and brains together. That it's useful to iteratively build exhibits to ensure that they communicate with the visitor.  That some topics are more popular than others.  That's it's very hard to tell exactly and precisely what's going to stick in any one individuals memory, or how they will use and interpret their experience.  And that there are a multiplicity of specialty uses.  That's easier to build an exhibit that doesn't engage people than one that does. That people will suffer to learn. That each exhibit is a mini-experiment.  That the museum visit is a very loose and unstructured event frequently spontaneous event, that it affects different people different ways and that's it's really difficult to measure an experience directly or longitudinally over the course of person's life.  This is a great list.
> 
> Museums are great places and valuable learning institutions, however it is unlikely that most informal learning for most people happens in museums.  Many of the observations and discoveries in the paragraph above, were previously known and not surprisingly correspond with many other aspects of human behaviors and learning.  The museum experience is brief and inherently complex.  While museums are valuable and significant components of our social fabric, it is unlikely that they will provide novel insights into the cognitive, affective and social nature of learning. It's likely easier to gain these insights elsewhere. Many aspects of the second paragraph above are summed by one metric, long term attendance.
> 
> The main reason to support the continuance and development of museums is not because they are pioneering new types of learning, or better types of learning, or even more effective learning.  They may do some of that.   Museums directly and broadly increase public accessibility to our understanding of aspects of the natural world and allow us to explore it.   Such are my observations and opinions.
> C
> 
> On Jun 27, 2012, at 8:03 AM, Carey Tisdal wrote:
> 
>> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
>> Incorporated, a worldwide network of science museums and related institutions.
>> *****************************************************************************
>> 
>> Jennie,
>> 
>> Yes, ideas of rigor evolve. Patton makes the point that the Gold Standard itself collapsed because of rigidity. In a nut-shell, when findings developed with strict control are "loosed" into the natural world with unlimited numbers of interacting variables which can effect dependent variables, then they don't hold up. They can also produce unexpected side-effects, some minor, but some major ones. The first of the references I posted below is a PowerPoint where Patton discusses these issues. It is a PowerPoint of a presentation he made. The other references are from the Zelik team.
>> 
>> Actually, Patton notes that drug testing goes through four phases (RCT's are Phase 4) and all four phases are needed for rigor (including the small sample uncontrolled correlational studies where effects are identified). The final phase involves RCTs. Actually, I was the one extending the argument that RCTs don't provide adequate rigor for drugs. I take a drug was put on the market without investigation of what happens when people stop taking it. It is safe to take (well, as far as anyone knows), but the withdrawal effects are quite substantial. There is no commercial impetus for investigation of withdrawal--actually, there are commercial biases that support NOT finding methods of for withdrawal. Patton does not criticize this area of drug trials. I do. I think the arguments for rigor posed by Zelig would have helped prevent a very daily effect of the application of science in my own life. :-)
>> 
>> If you will look on p. 84 of the first reference I posted you will find the rigor attribute model. This alternative model does not throw out traditional ideas of rigor. But remember when those ideas initially evolved, the wide-spread application of all sorts of research findings not part of the cultural context. Philosophers of science were making claims that the purpose of science is to produce knowledge and practical application should not be considered in judging quality or value. Culturally, in that era and in this one, this claim provides the freedom for exploration and scientific autonomy that researchers need. On the other hand, this is a claim can lead to ideas of rigor that are inadequate for the natural world contexts.
>> 
>> I really don't want to dismiss the implicit criticism of the earlier (and ongoing) remarks from Charlie and Eric about evaluation. But, I believe that the appropriate response is not, as they suggest, to stop doing evaluation but to get a better concept of rigor and use it-- to really look at our own work.  Those of us who do social science really need to raise questions when methods developed in physics and chemistry and medicine are imposed on us. These are higher prestige areas of science, and this itself is a cultural and historic artifact.  Certainly, scientific research in these areas have salient areas of application, particularly for warfare between nation states and in industry. Yet the methods which developed authentically from social science (including testing multiple-hypotheses against the evidence and  triangulation as ways to reach conclusions) may actually be more rigorous and provide stronger and more reliable evidence for decisions.
>> 
>> Jennie, I hope that clarifies my statements. A listserv is a difficult place to discuss rigor. But, the topic came up and a discussion of it requires some real thought and consideration. It is NOT a closed question and the concept does continue to evolve. I think this rigor attribute model is somewhat difficult to grasp--maybe that is a disadvantage of it. I am not advocating adopting it, but discussing it as an alternative and using the concepts that help us do better and more useful work.
>> 
>> Best regards,
>> 
>> Carey
>> 
>> Carey Tisdal
>> Tisdal Consulting
>> [log in to unmask]
>> 
>> 
>> 
>> 
>> 
>> On 6/26/2012 6:28 PM, Jennie Dusheck wrote:
>>> ISEN-ASTC-L is a service of the Association of Science-Technology Centers
>>> Incorporated, a worldwide network of science museums and related institutions.
>>> *****************************************************************************
>>> 
>>> This is a fascinating post, but I'm not sure I understood all of it--in particular the idea that rigor is an idea that is evolving. Carey, are you saying that applied science needs different standards of rigor than other science? Could you give an example of what criteria would be more appropriate to, say, a clinical drug treatment trial than a pre-clinical one?
>>> 
>>> Thanks,
>>> Jennie Dusheck
>>> Science Writing & Editing
>>> Santa Cruz, CA 95060
>>> 
> 
> The opinions and thoughts expressed here are my own and should in no way be construed or attributed to the Exploratorium or related organization, and do not represent an institutional position.
> Charles Carlson
> Senior Scientist
> exploratorium
> 3601 Lyon St.
> San Francisco, CA 94123
> [log in to unmask]
> Tel:   415-561-0319
> Fax:  415-561-0370
> http://blogs.exploratorium.edu/whyintercept/
> 


***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.

Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.

The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.

To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]