Document Type

Conference Paper

Publication Date

8-2022

Abstract

Peer review of scholarship is critical to the advancement of knowledge in a scholarly discipline. Despite this, scholars receive little or no training in effective and constructive peer review. The process of peer review has been routinely criticized in higher education for lack of quality reviews and reviewers, and reviews that are personal and not constructive. As a discipline, engineering education research (EER) benefits from diverse disciplinary backgrounds and perspectives of scholars, and as such relies on peer review of scholarship to generate, interpret, and translate knowledge. Supported by funding through the National Science Foundation, this project is developing, implementing, and assessing a project that conducts training in EER peer review for journal articles and grant proposals. This paper describes the Engineering Education Research Peer Review Training (EER PERT) project, which is designed to develop EER scholars’ peer review skills through mentored reviewing experiences. The overall goals of the EER PERT project are twofold: to establish, assess and evaluate a mentored reviewer program for 1) EER journal manuscripts and 2) EER grant proposals. In the first year of the project, two cohorts have participated in the mentored EER journal manuscript program through partnership with the Journal of Engineering Education (JEE), where triads (two mentees and one mentor) work collaboratively to review three manuscripts. Triads work with coaches informally to provide feedback on their experience and discuss manuscripts they are reviewing. Across the two cohorts, 42 mentees have collaborated with 18 mentors and 3 coaches, with some mentors and coaches participating in both cohorts. Project evaluation activities for the mentored manuscript review program included focus groups and exit surveys with both mentees and mentors in the first cohort. The main findings from the evaluation of the first cohort included the following themes: 1) the program provided valuable training and increased participant’s confidence in conducting EER scholarship, 2) the program fostered a sense of community and inclusion, particularly for those without EER backgrounds and from outside the US, and 3) program logistics could create barriers for participants, including working across time zones and tracking triad progress. Two main changes were implemented for the second cohort, including 1) a program dashboard for teams to track progress and organize update form submissions and 2) an optional monthly discussion meeting to supplement initial program orientation and triad meetings. The monthly discussions are intended to provide additional training in peer review topics and develop community among participants through networking opportunities. Planned discussion topics include updated JEE Author Guidelines and Review Criteria, inclusive language in scholarly publications, anti-racism in peer review, getting involved with the EER community, and roles and processes within the JEE Editorial Board. Program evaluation of the mentored manuscript review program is being leveraged to develop a mentored grant proposal review program wherein participants will work with former NSF program officers to review six grant proposals and participate in a mock review panel. This paper will present an overview of the journal manuscript and proposal mentored reviewer programs and findings from program evaluation.

Department

Integrated Engineering

Conference Name

2022 ASEE Annual Conference & Exposition

Conference Place

Minneapolis, MN

Download

Share

COinS