HR: 0800h
AN: ED51B-0408    [Abstracts]
TI: Field Test of a Peer Review System for Digital Geoscience Education Resources
AU: * Mayhew, M A
EM: michael.mayhew@comcast.net
AF: Synoptic, LLC, 42 Island Edge Drive, Ocean City, MD 21842,
AU: Hall, M K
EM: hall@scieds.com
AF: Science Education Solutions, 4200 W Jemez Rd Suite 301, Los Alamos, NM 87544,
AB: We report the results of an experiment aimed at developing a rigorous peer review system for evaluating digital geoscience education resources under consideration for acceptance into a digital library. The objective is the basis for developing a digital library of the highest quality that will become a trusted resource for Earth science instructors. Our model is NSF-style review panels. Panels were convened in locations having colleges, universities, and labs: Philadelphia, Boston, Austin, St. Petersburg, Seattle, Tucson, and Portland (Maine). The conveners traveled to the sites for the panel meetings. Panelists were reputable geoscience and geoscience education experts. The panelists proved to be uniformly supportive of the process and valued both the experience of serving on the panels and the introduction to resources they could take back to their classroom; thus, the process proved to have a valuable community-building component. The resources reviewed in the experiment, grouped by theme, were provided by the Science Education Resource Center at Carleton College; they are also catalogued in DLESE and NSDL. Panelists wrote reviews that addressed the criteria: 1) scientific accuracy and currency, 2) usability, and 3) pedagogical effectiveness. Additionally, the reviews addressed the questions: 1) Does this resource make an important contribution to Earth system education? 2) Would you recommend this resource to a colleague? and If you recommend major revisions for this resource, would you be willing to review it again? Each review concluded with a recommended action: 1) Accept, 2) Accept with minor revisions, 3) Accept with major revisions, or 4) Reject. Accept with major revisions requires a review by a subset of the panel conducting the original review. Once accepted, a resource enters a reviewed collection, with a specific indicator that it has been through the peer review process. Each panel was invited to single out those resources they considered particularly exemplary, which we refer to as a gold star rating; such resources will be indicated as such in the reviewed collection. Following the panel meetings, the conveners wrote summaries of the panel discussion for each resource; these are transmitted to the creator, along with anonymous versions of the reviews. The panel process proved to be a critical filter for the reviewed resources. Relatively few resources were accepted as is. In most cases, the majority of the resources were either designated as 1) Reject or 2) Accept with major revisions. Resources were most often rejected for their lack of completeness to be used in a classroom or they contained scientific accuracies. The review process modeled provides a basis for building a trusted geoscience education digital library of the highest quality on a less-is-more foundation, as opposed to the prevailing more-is-better philosophy.
UR: http://www.dlese- project.org/isovera/peerreview
DE: 0820 Curriculum and laboratory design
DE: 0825 Teaching methods
DE: 0840 Evaluation and assessment
SC: Education and Human Resources [ED]
MN: 2007 Fall Meeting