Assessment Methodology

General education assessment provides answers for internal and external accountability questions. It provides evidence of strength in general education skills and areas for improvement. The methods of general education assessment are a means to an end. That end is the improvement of general education through a well-documented and systematic process. The process of assessment at Northwestern Michigan College is well documented with the exception of a written methodology, which is the purpose of this document.

For accountability purposes, it is NMC’s goal that our degreed graduates (ASA, AAS, and ADN) achieve a sufficient level of accomplishment on the outcome capabilities. Sufficiency is defined by the outcome rubrics.

Institutionally, NMC measures achievement on the outcomes three ways:

1) with student work or artifacts (methodology)

2) with published nationally normed standardized tests (methodology)

3) with student perceptions of learning in locally developed instruments (methodology)

This triumvirate of methods is designed to demonstrate the full story of student achievement on institutional outcomes. This way the results of internally developed and validated methods are presented alongside externally validated methods. Suskie (2004) asserts that good assessment among other things, gives useful, accurate information, and is fair, ethical and systematized. We strive to meet these goals with our three methods.

All three methods are summative assessments of student achievement. Student work and accomplishment on standardized tests directly and objectively measure student performance. Moreover, the standardized test gives us the benefit of external evaluation. Student perceptions about learning the Graduate Follow-Up Survey indirectly and subjectively measure student achievement.

To achieve the goal of providing useful and accurate results, it is imperative that our methods measure what we are intending to measure (validity) and that when applied our methods provide consistent results (reliability).

The population to which we intend to generalize results for accountability includes our near-graduates (defined as those students with 52 or more earned college credits).

Direct and Indirect Measures of Assessment

Some DIRECT measures of student learning:

• capstone experience
• portfolio assessment
• standardized tests
• state or national licensure, certification or professional exams
• locally developed tests
• pre- and post-testing
• essay questions blind-scored by faculty across a department, discipline, or college
• qualitative internal & external juried review of comprehensive projects
• externally reviewed exhibitions and performances
• external evaluation of performance during internships based on stated program outcomes

*More than one direct measure of student learning in a given course, program, or general education requirement is recommended. Multiple measures, both quantitative and qualitative, ensure greater reliability.

Some INDIRECT measures of student learning:

• alumni, employer, and student surveys
• exit interviews of graduates and focus groups
• graduate follow-up studies
• retention and transfer studies
• length of time to degree
• graduation and transfer rates
• job placement data

*These should be used to supplement direct measures. Alone they are inadequate measures of student learning. Indirect measures, however, can support and enrich the information obtained through direct measures.

Some NON-MEASURES of student learning:

• grades and GPA (see below)
• instruments that are measures or reports associated with program review Examples:
  • Michigan Program Review of Occupational Education
  • curriculum review reports
  • evaluation reports of individual programs submitted by specialized accrediting agencies or committees of visiting experts
• information gathered not for assessment, but for specific administrative purposes Examples:
  • faculty publications and recognition
  • kinds of courses students select, including courseenrollments and course profiles faculty-student ratios
  • the percentage of students who study abroad
  • enrollment trends
  • the percentage of students who graduate with the baccalaureate in five years

Why are grades and GPA not considered measures of student learning?

Increasingly, grades and GPA's have been examined for their reliability and validity as measures of student learning. Although most institutions have not ceased using grades to report a student's final level of mastery in a course, many researchers, accreditors, and teachers have come to question whether grades really indicate what a student has learned in a course and what a student's level of ability is in a given area. Educators also have found that grades are influenced by numerous factors--institutional, cultural, social, personal, etc.--other than a student's actual ability. Final grades and GPA's are assigned in institutions dedicated to assessment practices; however, such schools also emphasize student learning as a process, not just a final product.

Of course, some grading methods can be more meaningful than others, and Primary Trait Analysis (Use of Rubrics) has become an increasingly valued method of assessing student work. For an excellent discussion of this topic, please consult Effective Grading: A Tool for Learning and Assessment by Barbara E. Walvoord and Virginia Johnson Anderson (Jossey-Bass 1998). This work is available in NMC's Center for Instructional Excellence.

Bibliography on Validity and Reliability

Alreck P. L., & Settle, R. B. 1995. The survey research handbook (2nd. Ed.) New York: McGraw-Hill.

Banta, T., & Associates. 2002. The scholarship of assessment. San Francisco: Jossey-Bass.

Gay, L. R. & Airasian, P., 1992. Educational Research: Competencies for analysis and applications (7th Ed.) Columbus, Ohio: Merrill Prentice Hall.

King, G., Keohane, R. O., & Verba, S. 1994. Designing Social Inquiry. Princeton, N. J.: Princeton University Press.

Mentkowski, M., & Associates. 2000. Learning that lasts: Integrating learning, development, and performance in college and beyond. San Francisco: Jossey-Bass.

Mentkowski, M., & Doherty, A. 1980. Validating assessment techniques in an outcome-centered liberal arts curriculum: Insights from the evaluation and revisions process. Milwaukee, WI: Office of Research and Evaluation. Alverno College.

Mentkowski, M., & Loacker, G. 1985. "Assessing and validating the outcomes of college." In P. Ewell (Ed.), Assessing educational outcomes, 47, 47-64. New Directions for Institutional Research. San Francisco: Jossey-Bass.

Mentkowski, M., & Rogers, G.P. 1988. Establishing the validity of measures of college student outcomes. Milwaukee, WI: Office of Research and evaluation. Alverno College.

Palomba, C. A., & Banta, T. W. 1999. Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.

Pike, G.R 2002. Measurement issues in outcomes assessment. In T. W. Banta & Associates, Building a scholarship of assessment (pp. 131-147). San Francisco: Jossey-Bass.