Part I: The Curriculum
9. Assessing the Outcomes of the Curriculum
Table of Contents




 

In its 1987 self-study, Colgate found itself enjoined by the process to do something it had, for some reason, not yet done—publicly articulate its mission. Now, ten years later, we find that that Mission Statement has provoked, in varying degrees of assent and qualification, discussions that have defined what we are more precisely to ourselves. The self-study process also requires an institution to review its mechanisms for assessing the outcome of its operations. This is a requirement that we address comfortably enough in most respects (see section 9 in Part II below) but that we find particularly difficult to do in terms of the curriculum.

A self-study with its focus on the curriculum is the obvious starting point, however, for an institutional effort trained on assessing that element. This effort should involve us in

  • thinking more habitually about our mission, goals, and objectives in relation to the curriculum
  • reviewing the ways we already assess the outcomes of the curriculum and formulating a plan to continue those efforts as well as to supplement them by other procedures
  • codifying procedures for ensuring that the results of all of our efforts at assessing the various parts—individual courses and programs as well as the whole—are communicated to those responsible for supervising them so that our assessment efforts result in improving the curriculum
  • disseminating what we learn—internally to ourselves, including to students, and externally, to alumni, parents, prospective students, employers, graduate schools, professional schools, and a general public increasingly vocal in its skepticism about liberal arts education.

Thinking about the curriculum in terms of our institution’s missions, goals, and objectives

As we have already noted in the introduction to this Part I of our report (and see also Part II, section 2, below), this self-study process has put our Mission Statement before the faculty and the community as a whole, asking everyone to comment on the degree to which it articulates the logic of the institution as they understand it. Throughout the process of interviewing faculty on a program-by-program, department-by-department basis, the steering committee especially urged that conversations include discussions of the goals of specific parts of the curriculum as they relate to our sense of mission. This experience has caused us often to reflect on a statement in the Commission on Higher Education’s pamphlet "Framework for Outcomes Assessment":

If a mission statement does not exist or has been ignored and forgotten, or is too generic to be taken seriously, or in other respects is no longer a helpful reality for the campus, the institution should develop a current and clear mission (p. 16).

Our mission statement, of course, as we note above, was not articulated until five years ago and since that time, until the steering committee of the self-study began its work, it was not widely disseminated in the various constituencies of the institution. While our conversations about it in those constituencies were very helpfully focused by the document we have, we also discovered in those conversations that many members of this community feel that we should strive to state our mission more elegantly. It is particularly important to reconsider our Mission Statement as the results of this self-study are incorporated into the regular workings of the institution. This process set out to make us more self-conscious, and it has done that. Moreover, discussions of new initiatives, as outlined in section 8 above, will be especially significant to consider in relation to what our Mission Statement says. Finally, we expect that the results of this self-study, in many areas, will lead to clarification of what is now only implicit in our Mission Statement—the goals and objectives of the college. (See Part II, section 2 below.)

Assessment procedures for elements of the curriculum

Some aspects of the curriculum are already formally assessed.

  • Students complete Student Evaluation of Teaching (SET) forms for each course offered in the curriculum. Study groups are evaluated by students using a different and more extensive form. (See SD 39, evaluation instruments.) The form asks students to describe why they took the course and what they learned from it as well as for their comments on the quality of the instruction. SET forms are reviewed, when grades for the course have been submitted to the registrar, by the teaching faculty member and his or her chair as well as in personnel reviews (at the third year, at the sixth year or tenure time, and in subsequent promotion reviews). These reviews all involve other members of the candidate’s department reading SET forms as well as the Dean and her advisory council. What is learned from SET forms thus has considerable impact on the design of courses, on pedagogical strategies, and on the way courses are arranged relative to one another in the curriculum.
  • Departmental and programmatic curricula are evaluated by external consultants periodically (in a 10-year cycle). Since the beginning of this self-study especially, the reviews that are taking place have been sharpened by the institution-wide conversation ongoing about our mission and about the curriculum as a whole in relation to that mission.

  • We found in our conversations with departments and programs that some parts of the college have the means and inclination to track the results of their curricula in a number of ways after students are graduated. In some cases, this is done by collecting data on how students perform on standardized tests. External reviews of some of our language departments have particularly recommended such procedures in that area. Graduate school admissions and job placements are also collected.

Departments and programs should designate a historian to keep track of graduated concentrators. The person in the department or program who serves this function should be known for this function by students when they are here, and department budgets should include funds for initiating contact with graduates.

In some fields, however, students are out of the institution several years before they join the graduate program or profession that will define them in their adult lives. One department (see SD 39) surveys its graduates several years out of Colgate.

Departments should consider what shape a survey of their alumni concentrators would ideally take. They should also consider conducting such a survey.

The steering committee noted that the Office of Career Services can now retrieve information about graduates more comprehensively and effectively for departments and programs.

The Office of Career Services should continue to develop procedures for gathering information about the patterns our graduates follow.

The required form of annual reports should continue to solicit this information. (See SD 39.)

  • The Core Program holds an annual review of the program in all of its parts that is attended by large numbers of faculty (last spring, over 100). SET forms about the various courses as well as particular instruments used by specific parts of the program are reviewed by the leadership of the Core, and summaries of their findings are used to shape conversations about pedagogy and the content of courses. The emerging components of distinction and high distinction are clear opportunities for faculty to assess the outcome of the four required courses that students must complete before they enter those parts of the program.
  • Most departments and programs require concentrators to take an advanced-level course in which a major paper or project appropriate to the discipline is required. The steering committee found that faculty talked in terms of their experiences of these courses, as well as in terms of their experiences with honors work, when asked about the outcomes of their programs. This assessment is not formally collected and codified; but above (in section 8.C) we have recommended that faculty go about it more concertedly.
  • The Dean of the Faculty now reports to faculty some elements of student performance (the average grade and various graduation statistics). Elsewhere in this report we have recommended that other data be collected and shared. (See above, section 4.)
  • A first-year survey (SD 44) and senior survey (SD 63) are currently used at Colgate, the latter; however, is not effectively administered.

The Dean of College’s office should refine the procedure for administering the senior survey so the data we gather from it can be statistically significant.

  • Colgate will also begin administering a comprehensive alumni survey to graduates who are out five, ten, and twenty years. This survey, which was developed by the Higher Education Data Sharing Corporation (HEDS), has been used by approximately 20 other colleges. It will enable us to compare the experiences and evaluations of our graduates to those being graduated from peer institutions. A major focus of this survey is on the academic experiences of students, including coursework in and outside of their concentrations, internships, independent study, as well as the skills, knowledge, and abilities that were enhanced by their education (e.g., quantitative and technological literacy, foreign language use, analytical thinking, moral and ethical development). The survey will also collect information about post-baccalaureate education, career choices, and demographic details.

These are the major elements of curricular assessment currently in place. Others may be more appropriate, however, and a procedure for considering future development in this area is discussed below.

Making sure that assessment has an impact on curricular planning

There is good evidence that what assessment we currently do is incorporated into local decisions about the curriculum.

  • Most faculty are very attentive to what they learn from SET forms. Statements that candidates for tenure and promotion write for their cases not only reveal considerable thoughtfulness about what these forms communicate about what students learn and the way they learn; they also show faculty to be responsive to student comments on the content of courses and even the overall value of the material in the course.
  • Departments and programs are likewise attentive to what they learn through aggregate SET forms. This emerges also in the input of departments and programs into personnel cases. They are also carefully responsive to the reports of outside reviews. Departments who pursue statistics about student outcomes after graduation make what they learn the provocation for curricular and pedagogical review.
  • In both its roles of making personnel decisions at the third year review and approving courses and programs, the Dean’s Advisory Council shows itself to use the results of curricular and pedagogical assessment to make decisions about the resources allocated to various parts of the curriculum and about the increase, reduction, or addition of its various parts.

It is at the more global level of the DAC that the steering committee suggests we locate our resolve to be more systematic about the assessment of the curriculum.

The DAC should familiarize itself with resources in the field of outcome assessment. Acting as a collective body, it should formulate a plan to improve and increase how we currently assess outcomes of the curriculum. Acting individually in relation to the departments and programs in their respective divisions, members of the DAC should assist departments and programs to refine their efforts. Within the Academic Affairs Board, the DAC should introduce the concept of assessment into faculty-student discussions of curricular matters.

The Academic Affairs Board should consider how to analyze the curriculum, particularly the required curriculum, with respect to the specified outcomes the college sets its curriculum to achieve.

The value of the function that the DAC serves in our curricular governance is particularly clear with respect to assessment.

Disseminating what we learn from assessment procedures

Liberal arts faculty are often reluctant to use the language of outcomes and assessment. Perhaps there is some defensiveness and snobbery in this, but it may also have more respectable causes. It is probably true that the least valuable things we could do for our students are the easiest to measure; and conversely, it is difficult even to articulate let alone assess what we value most in the educational process.

Nonetheless, seeing the curriculum in terms of outcomes, including but certainly not confined to the content of the various courses that we teach, is vital to both the process of what we do and the way that process is viewed. Students need to internalize the self-confidence to which their education entitles them to take themselves seriously as students. They also need to be able to communicate its value to prospective employers, parents, and others who control access to opportunities they wish to pursue. Increasingly, colleges also need to articulate the more broadly focused outcomes of curricula themselves to students and parents, to alumni, to the government and foundations. Talking about the outcomes we strive to achieve can also focus curricular discussions in ways that can get beyond controversies over the relative value of the subjects and methods that define the academic disciplines. The steering committee certainly appreciated this result of thinking in terms of outcomes in the course of this self-study process.

Specifying outcomes

We seem to talk about outcomes in terms of three elements of the life our students live when they leave here: work, citizenship, and personal satisfactions. Although these categories overlap, each embraces elements that seem less proper to one or both of the others. We list, under each category, the outcomes of the curriculum we think are most related to each one.

Work-related outcomes of the curriculum

The following are career attributes addressed by the curriculum that are more general than mastery of particular bodies of information:

  • Writing and speaking, communicating ideas effectively
  • Thinking, seeing logical connections, identifying what is important and what is not, anticipating cause and effect, uncovering the subtle, dealing with abstraction
  • Knowing where and how to find information, evaluating sources
  • Quantitative confidence, numerical sense, finding a story in data and telling it
  • The ability to work on long-term projects, independence, organization, teamwork, persistence
  • Being able to see through the eyes of others, appreciating differences in viewpoints and cultures, communicating in another language
  • Technical knowledge

Citizenship outcomes for the curriculum

Many of the career attributes listed previously are also important for graduates in their roles as members of society, especially thinking, information literacy, quantitative confidence, and seeing through others’ eyes. To them, we add

  • Historical perspective
  • An understanding of social institutions: education, government, law, medicine, the economy
  • An awareness of the forces that govern relationships among nations and cultures
  • An appreciation of the complexity of leadership
  • A sense of the social value of civic efforts, including volunteer efforts
  • A profound appreciation for how the physical and biological components of human life have implications for local, national, and international communities

Personal Outcomes of the Curriculum

These outcomes are the hardest to track and, perhaps not coincidentally at a liberal arts college, for many students and faculty, they seem the most important and the most elemental to the kind of school we are. The music course that led to a life-long passion for chamber music, the art course that is remembered on every trip to a museum, the politics course that makes one read the New York Times OpEd every day, the science course that illuminates the reports from a spaceship on Mars, classes in philosophy or literature or art that raise issues a young adult can barely grasp but which take on deeper significance with maturity: all of these may have a profound influence on a student’s later life. Our curriculum demands and rewards qualities that lead students to these experiences.

  • Exposure to many kinds of beauty and ways of knowing
  • Habits of intensity and intellectual vigor
  • The capacity to think ethically
  • Respect for what one does not understand
  • An awareness of the personal/emotional rewards of being responsive to others
  • An interest which enriches a whole life
  • Self-confidence grounded in humility and a willingness to learn, change, and grow

As we consider how to assess the degree to which the curriculum brings about these outcomes in students, we should take advantage of procedures and instruments we already have even as we consider devising others. We may wish to track information that is quite specific. For example, while it is imprecise to measure outcome by assuming that students learn everything that is taught in a given course, some assessment of the degree to which our curriculum can claim to have certain outcomes can by gathered from the work that students produce in the required curriculum. An instrument would probably have to be devised to survey faculty about their courses in all parts of the curriculum: amount of writing required, numbers of presentations, use of library, number and nature of long-term projects, inclusion of units on ways to retrieve information and other technological skills, work in a foreign language and attention to other cultures.

Our sense of the outcomes of the education we offer is something to share with students and their parents.

First-year orientation should make students more self-aware about the outcomes of their Colgate education, especially those concerning citizenship and life-enrichment. Senior events such as the "Real World" series should also address volunteering, local government, supporting the arts, connecting one’s education with one’s later life. Events for alumni or parents should encourage participants

to think of the outcomes of a Colgate education in all dimensions of a student’s life.

We should take care to keep the conversation focused not only on specific skills but on those skills as means to a larger end. These will be new kinds of conversations for us at Colgate. As we continue to strive to distinguish ourselves as an institution, they are particularly important conversations to have with our students, while they are here and after they are graduated.



Part II: Standards of Excellence></a><br>
	<a href=Appendices
Search
Next Page