ISEN-ASTC-L is a service of the Association of Science-Technology Centers
Incorporated, a worldwide network of science museums and related institutions.
*****************************************************************************


U.S.STUDENTS ANDSCIENCE: AAAS TESTING GIVES NEWINSIGHT ON WHAT THEY
KNOW ANDTHEIR MISCONCEPTIONS
 
The American Association for the Advancement of Science has launched an
innovative website with more than 600 multiple-choice test questions to
help educators assess more precisely what students know about key ideas
in science and -- just as importantly -- the incorrect ideas they have.

 
The new website, which also offers an unusually detailed picture of how
middle and high school students across the United Statesare currently
doing in science, features information on what they know and on hundreds
of misconceptions they have about everything from the size of atoms to
whether all organisms have DNA.
 
Knowing these misconceptions and how pervasive they are -- which is not
typically part of the analysis of test results from state testing or
from leading national and international testing organizations -- can
help teachers improve instruction and better design their own test
questions to assess whether students truly understand the science
concepts they are being taught.
 
The newly developed test questions also counter the widely held view
that multiple choice questions are useful only for testing recall of
memorized definitions and trivial facts, says George DeBoer, deputy
director of Project 2061, the AAAS program that developed the new
assessment website.
 
"As a result of our efforts, many of the test questions included in the
new website measure not only knowledge of factual information, but they
also probe a student's ability to explain real-world phenomena, reason
logically through problem situations, or identify the reason why a claim
is true," DeBoer said.
 
Multiple-choice testing has some clear advantages, he said. It tends to
be less costly and easier to administer and score than other types of
testing. "It is unlikely to disappear from the assessment landscape," he
said. "The goal of the project, then, was to improve on the design and
use of multiple-choice questions."  
 
But the focus on multiple-choice testing should not be misinterpreted
as an endorsement of it as the best approach to student assessment,
DeBoer cautioned. "Any type of assessment can be used effectively," he
added, "as long as attention is paid to certain fundamental design
principles."   
 
The new AAAS website (http://assessment.aaas.org) presents detailed
information on how a national sample of students answered each question,
along with an analysis of both their correct and incorrect responses. In
contrast, national and international assessment programs such as the
National Assessment of Educational Progress (NAEP), Trends in
International Mathematics and Science Study (TIMSS), and Program in
International Student Assessment (PISA) report on the percent of
students in participating states or countries who answer items correctly
but do not report on the incorrect ideas students have, DeBoer said.
 
Project 2061, founded in 1985 by AAAS to improve science education,
developed the assessment items and collected data on them under a grant
from the National Science Foundation. The website, where the results of
this work can be found, includes data on student performance on the test
questions by gender, grade level, and whether or not English is the
student's primary language. Each question typically was answered by at
least 2000 students in field tests involving school districts across the
nation. In 2010, for example, more than 90,000 students in 814 schools
participated in the field tests. Project 2061 researchers also conducted
on-site interviews with students to gauge the effectiveness of the
questions.
 
Constructing good multiple-choice questions requires knowledge of
science and of effective assessment practices. "Too often," DeBoer said,
"test questions are not linked explicitly to the ideas and skills that
the students are expected to learn." And too often the questions are
written in a way that makes it difficult for students to understand what
is being asked, he said. 
 
With increasing calls for national science standards and a common core
curriculum for middle-school and high-school students, it is more
important than ever to get assessment right, DeBoer said. "Good
assessments can be used to actually improve students' learning and not
just to hold teachers and schools accountable," he added. "Assessments
that are designed to diagnose students' misconceptions can be powerful
educational tools." 
 
For the key idea that "all matter is made up of atoms," the AAAS
website notes that 27% of the middle school students and 20% of the high
school students who were tested incorrectly believed that "cells are not
made up of atoms."  On the key idea that "genetic information is encoded
in DNAmolecules," 40% of middle school students and 30% of high school
students had the misconception that only animals have DNAwhile plants
and mushrooms do not.
 
Having information about such misconceptions is just as important as
knowing how many students answered a question correctly, DeBoer said. It
can reveal gaps that prevent a coherent understanding of the topic.  
 
The AAAS website is organized to quickly give users a picture of what
students know and the misconceptions they have. The test questions for a
specific science concept are listed together, arranged from the highest
to lowest percent correct.  The misconceptions also are listed in order,
with the misconceptions selected most frequently at the top of the list.

 
So, for example, there are 20 questions probing the idea that "genetic
information is encoded in DNAmolecules." At the top of the correct
responses, 94% of middle school students knew that "DNAis the molecule
that contains genetic information that is passed from parents to
offspring." At the bottom of the list, only 15% of the students knew
that "Genes are chemical sequences of nucleotides, not the actual
expression of characteristics or traits." 
 
"This is extremely valuable information for teachers and curriculum
developers to have" said DeBoer, "because it shows them where
instruction needs to be targeted."
 
The website has received early positive reviews from teachers and
curriculum specialists who attended a recent workshop at AAAS on the
methods used to develop the test items.
 
The emphasis on student misconceptions is particularly helpful, said
Deagan Andrews, a curriculum and assessment specialist in the Greeley,
Colorado, school district. Referring to some of the major standardized
test programs, he said: "No one releases any information about
misconceptions. They are interested in whether students got it right or
wrong."  He said the emphasis on misconceptions "is a critical piece
that has been missing," offering teachers an opportunity to "think about
their instruction and what they may be doing that may be perpetuating
misconceptions."
 
"Students create strange conceptions about the world from their
experiences," agreed Anu Malipatil, a school administrator for a network
of charter schools in New Yorkand Connecticut. "It becomes more
difficult to teach students without actually addressing the
misconception first."  She said most state tests do not provide an
analysis of incorrect answers. The AAAS test items, she said, can be
used by teachers to diagnose potential problem areas and find out how
many students in a classroom have misconceptions about key concepts.
 
Teresa Eastburn, a former classroom science teacher who is now an
education and outreach specialist at the NationalCenterfor Atmospheric
Research in Boulder, Colorado, said the assessment site ultimately
benefits teachers as well as students, particularly teachers who may not
have an extensive background in science. Because of the level of detail,
including the emphasis on misconceptions, the site "will really inform
teaching," Eastburn said.
 
# # #
 
The nonprofit AAAS is the world's largest general scientific society.
For more information, go to: www.aaas.org.
 
AAAS Project 2061 offers technical assistance in the area of diagnostic
assessment development to schools and school districts, as well as other
educational programs, on a fee-for-service basis. In addition, a
three-day workshop on Project 2061*s approach to science assessment is
offered several times each year.
 
For more information, please contact Mary Koppal at [log in to unmask]
 
 

***********************************************************************
For information about the Association of Science-Technology Centers and the Informal Science Education Network please visit www.astc.org.

Check out the latest case studies and reviews on ExhibitFiles at www.exhibitfiles.org.

The ISEN-ASTC-L email list is powered by LISTSERVR software from L-Soft. To learn more, visit
http://www.lsoft.com/LISTSERV-powered.html.

To remove your e-mail address from the ISEN-ASTC-L list, send the
message  SIGNOFF ISEN-ASTC-L in the BODY of a message to
[log in to unmask]