A Longitudinal Student Outcomes Evaluation of the
SUNY - Buffalo State Undergraduate Summer Research Program
At Buffalo State we have undertaken a multi-year effort aimed at developing and field-testing a methodology for measuring student learning and related student outcomes for the Undergraduate Summer Research Program. This effort is being coordinated by Dr. Jill Singer, Director of Buffalo State’s Office of Undergraduate Research, in collaboration with Daniel Weiler Associates, a professional evaluation firm with many years of experience evaluating NSF-supported educational programs. The purposes of the evaluation are to: 1) obtain a reliable assessment of the program’s impact on participating students that goes beyond the student survey data that has so far been the principal source of impact findings reported in the literature; 2) obtain information from faculty mentors on how the program has influenced their teaching and their view of student capabilities; 3) provide information to participating students that helps them assess their academic strengths and weaknesses; and 4) begin the creation of a longitudinal database that can provide impact data on a range of outcomes for different types of students and a variety of academic disciplines.
Working with faculty from a wide range of disciplines (including arts, humanities, and social sciences, as well as STEM faculty), the evaluation has selected 11 student outcomes to be measured. A detailed rubric describes the specific components of interest for each outcome, and faculty mentors assess students on each component, using a five-point scale. Students evaluate their own progress using the same instrument, and meet with the faculty mentor to compare assessments as a way to sharpen their self-knowledge. A range of complementary instruments and procedures rounds out the evaluation. A preliminary version of the methodology was field-tested with a small number of faculty mentors and students during the summer of 2007 and a refined evaluation has been implemented during the summer of 2008 with the participation of all of the program’s faculty and students (17 mentor-student teams).
A CUR Quarterly article (Spring 2009) describes the approach we are taking to assess our summer research program and how we are measuring outcomes that are related to the experiential goals of the program. A CUR Quarterly article (Spring 2012) provides data for each outcome and a summary of our findings for the period 2008 - 2011.
Download the 2009 article and the 2012 article. Thank you to the CUR Quarterly editor and the CUR National Office for giving us permission to access the two articles.
Overview of Buffalo State's Evaluation Effort (.pptx slides presented during CUR's 2010 National Conference)
Findings from Buffalo State's Evaluation Effort (.pptx slides presented during CUR's 2012 National Conference)
Links to static versions of the survey instruments are provided below:
Revised forms based on findings to date:
- Elimination of ‘confidence’ score from all three student questionnaires and replaced by a single question at the end of the survey that asked students how certain they were about their skill level and responses and why.
- Elimination of ‘confidence’ scores from mid-summer report for mentor (retained in pre- and post-research assessments).
- Simplification of questionnaires so that optional outcomes identified at end rather than after each outcome category.
Candidate Questions for Mentor's Student Intake Interview
Pre-Research Assessment - Student
Pre-Research Assessment - Faculty Mentor
Mid-Summer Project Report - Student
Mid-Summer Project Report - Faculty Mentor
Final Project Report - Student
Final Project Report - Faculty Mentor
