Print Page   |   Contact Us   |   Report Abuse   |   Sign In   |   Register
TL v56n2 The Merits of Merit
Share |
 

 

Tennessee Libraries

Volume 56 Number 2

 2006

 

The Merits of Merit: Developing a Merit-based

Library Faculty Evaluation Procedure at

Northern Kentucky University

Perry Bratcher
Library Systems Manager

Lois Schultz
Head of Technical Services

 Northern Kentucky University

 

Conference Abstract: This presentation outlines the issues involved in the development and implementation of a point-based peer-review process for scholarly/service activities used in conjunction with the supervisor's job performance evaluation.


Background

Faculty status for librarians has been a topic which slowly developed during the first part of the 20 th century, gathering steam by the late 1950’s and subsequently becoming a popular issue beginning in the 1970’s. During this process it was important for librarians to align their activities as closely as possible to the teaching faculty. Due to the nature of the librarian’s job, a compromise between the description of duties between librarians and teaching faculty was necessary. This led to differing descriptions in the area of primary responsibility. For teaching faculty, criteria were outlined under a general classification of “teaching effectiveness”. For library faculty, this area was generally entitled “job performance”. Other areas of library faculty responsibilities which are similar to the teaching faculty are scholarly activity, university service, and community service. These three areas are more closely aligned than the primary area of responsibility.

Research on the duties of faculty librarians in several publications resulted in the descriptions of these activities. The vast majority focused on the descriptions of the activities rather than a performance assessment. The SPEC kit entitled Performance Appraisal in ARL Libraries (1979) documented procedures from six libraries. The performance criteria in these documents were either a narrative evaluation by the librarian and/or supervisor in each area (job performance, scholarly activity, etc.) or a ranked scale (excellent, good, etc.) under several defined criteria (quality of work, quantity of work, etc.). A listing of activities was made under non job performance areas. In a related SPEC Kit published in 1992 entitled Academic Status for Librarians in ARL Libraries, the case studies showed that libraries were providing more detailed examples of activities under the non job performance areas and relating these more closely to promotion and tenure. However, data outlining the year-to-year evaluation of performance in these areas were lacking.

This paper presents the authors’ experiences in helping to develop a detailed performance evaluation procedure at Northern Kentucky University for the areas of scholarly activity, university service, and community service. The process and issues for developing this evaluation method are outlined, along with subsequent issues resulting from the first two years of implementation. The issues discussed will provide ideas for other library organizations dealing with performance assessment issues for library faculty.

Site Information and Initial Charge

Northern Kentucky University (NKU) is a mid-sized academic institution serving 14,000 students in the Cincinnati metropolitan area. The University provides associate, bachelor, selected master degree programs, and a Juris Doctor through the Chase College of Law. NKU has two libraries – W. Frank Steely Library and the Salmon P. Chase College of Law Library. The focus of this paper is the performance review process for Steely Libraries 18 librarians, two of whom are non-tenure track. Steely Library employs 18 librarians, 2 of whom are non-tenure track. Librarians are evaluated annually.

In September of 2003, the Associate Provost for Library Services (APLS) charged the Steely Faculty with naming a task force to develop a system for the peer review of scholarly/creative and service activities. This change was necessary for four primary reasons: 1) the implementation of merit only salary increases by the University President, 2) the need for a change in the process cited by the APLS who was hired in the fall of 2001, 3) library faculty members’ desire for improved equity in the performance evaluation process, and 4) the emphasis by the University President that faculty performance assessment be based on the institution’s stated mission of instruction, research, and service. The charge by the APLS required the system to implement the following elements:

  • A graduated set of merit rankings
  • A method of evaluation that would result in the quality, quantity and significance of activities represented in a way that facilitates comparison of activities and results in a meaningful ranking
  • An evaluation method performed by a subgroup of library faculty with periodic rotation of membership
  • A process that will take place immediately following the performance review process for job performance done by the supervisor
  • A process that would forward recommendations to the Associate Provost for Library Services, who would have final decision-making authority

In summary, a librarian’s annual review would be a two-part process: one based on the primary assignment and the second a peer-review ranking by a subgroup of library faculty. In addition, the task force should deal with the concepts of changing job emphases and priorities.

Task Force Development and First Semester Tasks

A task force of six librarians volunteered in September 2003 to work on the performance evaluation charge. This task force was made up of tenured and non-tenured librarians from various library departments, as well as supervisors and non-supervisors. At the first meeting in early October, a semi-monthly meeting time was established. It was also decided that research into the matter was required, which took a month to complete. The task force decided to research benchmark institutions, other Kentucky institutions, regional institutions, other NKU departments and library literature. This research yielded the following responses: one benchmark institution, four Kentucky institutions, three regional institutions, three NKU departments, and two SPEC kits. The task force felt this amount of information was sufficient to begin work, primarily due to the fact that there was a lot of variety in the responses. These responses ranged from purely narrative to the assignment of values in distinguished categories. Some sites included various levels of peer evaluation while others included evaluation from the supervisor alone. Of those evaluated by peer groups, membership was sometimes purely voluntary and sometimes by appointment only. Activities listed under the non job performance areas were also varied. Some institutions also included mathematical formulas for determining salary increases, from the simple to the complex. The task force felt all of this information helped them to consider a number of possibilities and to think outside the box. During this time three issues surfaced which needed resolution: 1) whether procedures for tenured and non-tenured librarians be the same (to align with the reappointment, promotion and tenure process), 2) how sabbaticals and other leaves be treated, and 3) what the process for non-tenure track faculty should be.

Using the research cited above, the task force made considerable progress by the end of the year. First, the task force made some “basic assumptions” which consisted of the following:

  • Scholarly/service area evaluation will be performed by a peer review committee with the supervisor evaluating job performance (NOTE: a later charge by the APLS required the task force to develop a brief procedure for the evaluation of job performance)
  • Individuals must perform satisfactorily in job performance in order to be considered for merit (in order to keep the main focus on job performance)
  • There would need to be a different method of evaluation for non tenure track faculty
  • A calendar of activities needed to be developed for yearly consistency
  • A method for the development of graduated rankings needed to be developed in order to avoid clustering of merit raises too narrowly
  • The Chair of the new peer review committee would sign off on all evaluation forms
  • The evaluation period would cover one year unless change was warranted (e.g. year of no merit money)

By the end of the year, the task force had also made some decisions regarding the methodology for performing the peer evaluation. These decisions included the following:

  • The use of the current scholarly/service form as a template for the development of a point-based evaluation system. This template divided activities by various types (as outlined in the final document) including areas such as publications, holding office in professional organizations, conference presentations, etc.
  • The structure of the peer review committee was finalized: five members drawn from a lottery of tenure and non-tenure track with the chair of the group elected by this group. The Chair of the previous year’s committee would serve as an ex-officio. When a member of the committee was being evaluated, he would leave the room.
  • Non-tenure track faculty would be evaluated primarily in the job assignment area with “consideration” given to the other areas.
  • Publication activities could be in areas outside of library science in order to give faculty the opportunity to draw on their subject expertise.
  • Joint authorship/presentation would be rated the same as sole authorship/presentation.
  • Assignment of point values (major and minor) were made with consideration given to balancing these values between categories (how did the point values assigned to writing an article align with a presentation at a national conference, etc.).
  • Consideration was given to make sure decisions agreed with the campus faculty handbook.

Second Semester Activities

At this point, the task force felt it should get feedback from the library faculty in order to make sure that all issues were considered. A draft document was distributed via email and library faculty members were given a chance to respond. As a result, twenty-three areas of concern arose, some major and some minor. The major areas of concern were how to address a shift in job duties, concerns over faculty being evaluated by peers rather than their supervisor, the need for a grievance procedure, alignment with the reappointment/promotion/tenure process, and how to account for year to year variances. These issues were considered by the task force and revisions were made. Once made, the task force needed to develop a method to finalize the document. It was decided that a full meeting of the library faculty was needed to discuss the issues. Once this meeting took place, the task force would take suggestions and consider a final revision of the document. Once this revision was made, another presentation would be made to the faculty for a vote (no discussion). The final document addressed the following major elements:

  • Reappointment/promotion/tenure and merit evaluations are separate processes
  • The merit document is a “living document” open for future revisions
  • An appeal procedure for performance evaluation is already outlined in the NKU Faculty Handbook
  • The number of merit “tiers” and number of faculty in each tier can vary from year to year
  • The supervisor evaluation form consists of a point range from 1-50 with 30 being considered “satisfactory”
  • The APLS will pull money off the top for non-tenure track faculty and “special circumstances”
  • Non-print publications are rated the same as print publications

Prior to final approval by the faculty, the task force performed a “test run” of the evaluation process by applying the procedures to their previous year’s activities. As a result, discussion led to a minor shifting of point values. The final document was approved by the faculty and APLS in late spring 2004 for implementation in the spring 2005 evaluation (covering the 2004 calendar year). Final approval was given by campus legal council and the Provost in October of 2004.

First Year Implementation

The first step in implementing the peer review merit process was the selection of the merit review committee. According to the calendar (Appendix A) this committee would be determined at the May faculty meeting. The committee selection was postponed until June to allow for approval by the provost and legal counsel. Confident that the instrument would eventually be approved by all parties, faculty members proceeded to select a committee at the June faculty meeting

The composition of the first review committee helped smooth the process of the review, even though committee members did not recognize it at the time. Three members plus the past chair had been on the task force that had designed the process; thus, they were very familiar with the instrument. Four of the members plus the past chair had been at NKU for ten or more years, had tenure, and had served on tenure review committees. Although different, the tenure and merit processes had many similarities; thus, those members had a basic knowledge for assigning merit points.

In March the intense review process started. It took over two hours for the committee to review the first individual, even though the points for activities were already determined (Appendix B). The committee only had to decide major or minor points for the activity. So why did it take so long?

First, the members who had not been on the task force had to be oriented to the process. In several instances long discussions of the task force were reenacted. Activities were compared to other activities and questioned as to whether they carried equal, less, or more value. The struggle between time and prestige appeared again. Should point value be assigned based on the amount of time an activity took or on the prestige of the activity? For instance, should teaching a semester long class which often takes significantly more time than an article be valued equal, less than, or more than an article which is considered by many to be more prestigious? At times it seemed like every activity was going to be revisited and rehashed again.

Orienting the members that had not been on the task force was only one of several factors that caused the review to be slow. The task force had left the decision of assigning major or minor points of an activity to the review committee. The merit review handbook simply stated that “criteria for determining major or minor value may include length, place of publication, content, etc.” This led to lengthy debates. Is a one paragraph review in a prestigious journal worth more points than a two page review in a state publication? Besides the difference in status of the publication, some contend that a one paragraph review is harder to write than a two page review.

The task force had repeatedly stated that every activity would not be included in the list and that the review committee would have to make decisions for the moment. After the review process was completed, these activities could be discussed and voted on at a faculty meeting. This proved to be the case. There were several activities that the task force had not included on the list. Some activities that were not on the list included the following:

  • Supervising a graduate assistant
  • Self-published works
  • Frequent contributions to a listserv
  • Being interviewed by a newspaper
  • Being a reviewer for a journal

There were also unexpected issues that emerged. Some of these included

  • Appointed to a committee that never met
  • Incorrect dates
  • Missing items (known because a committee member had participated in the same activity)
  • Teaching more than one section of a course
  • Giving the same presentation twice
  • A committee assignment as a result of a primary assignment
  • A presentation as a result of a primary assignment
  • Preparing a display
  • Membership in a section of an organization

The issues around teaching warrant further explanation. Teaching a semester long class has a high point value. The committee was shocked to discover that some individuals had taught four or five sections of a course during the year. That one activity gave them close to a hundred points. Should teaching subsequent sections of a course have the same point value as teaching the course the first time? This was an issue that the faculty should revisit before the next review.

“Committee assignment as a result of primary assignment” and “Presentation as a result of primary assignment” are also complex issues. For instance, since the Head of Technical Services is automatically placed on the Collection Management Committee as a part of the job, should this committee count for service points? Likewise, the Extended Collections Services Librarian is responsible for making sure reserve items are within copyright compliance. If he/she gives a presentation on copyright to an academic department, is that part of the primary assignment?

The task force had not wanted the merit review process to become as complex as the tenure review process. As a result, documentation and explanation were not asked for, as large binders of materials were not wanted for review. However, the merit review committee decided they did need a sentence or two of explanation on some items.

The review committee tried to be as consistent as possible. This often meant going back and reviewing someone’s points when a similar activity arose on another individual’s form. The committee did contact individuals by email and ask for explanations when an activity was unclear. By the end of the process there were several issues that the faculty needed to discuss and vote on before the second evaluation.

Distribution of Merit Dollars

After the points were assigned, the remaining steps in the process (Appendix C) were just simple math calculations. The tier rankings for the first year were as follow:

  • Tier 1 had two individuals with a point range of 178-199
  • Tier 2 had 5 individuals with a point range of 147-165
  • Tier 3 had 5 individuals with a point range of 112-132
  • Tier 4 had 3 individuals with a point range of 79-84

Revisions After One Year

The merit review committee compiled a list of items to be reviewed and voted on by the faculty before the next review. Some of the items included reexamining the points for teaching and grants, having faculty write short explanations of activities that would not be known by other library faculty members, and adjusting the calendar so the merit form is submitted to the supervisor at the same time as the primary assignment review form. Merit review items were on almost every faculty meeting agenda during 2005. As items were discussed, new items were generated. One of the new items was a name change for the review committee. The committee is now called the Scholarly and Service Review Committee (SSRC).

Appendix D contains the point values for new and changed activities.

Second Year Review

The review process just ended for the second year. Neither of the authors was on the review committee in year two, but from talking with committee members some insights have been gleaned. First, the committee had to revisit and rehash more activities than the first review committee. This most likely was a result of only two individuals having been on the task force and only the past chair was on the previous review committee. Second, the revised point value for teaching is working well, but the grants area, as well as some other activities, still needs to be reviewed.

The tier results for year two are below, but if one compares it to year one, it must be noted that an adjustment was made in the points for primary assignment in order to balance the three areas. In year one the point value for primary assignment was 0-50. In year two the point range for primary assignment was 0 to 100.

  • Tier 1 had one individual with 234 points
  • Tier 2 had 5 individuals with point range of 192-214
  • Tier 3 had 3 individuals with point range of 174-182
  • Tier 4 had 2 individuals with point range of 143-156
  • Tier 5 had 2 individuals with point range of 129-131

The evaluation instrument needs to reflect the values of the university. It has been stated that the instrument is a living document; thus it should change as the values of the university changes. This appears to be happening. The president of the university is pushing for public engagement, so an additional line of points was added under service.

The associate provost for library services wanted the productivity in scholarship and service to increase. The general feeling of the faculty is that this is happening; however, since the primary assignment point range was increased, one can’t tell from comparing the point range in the tiers. The authors do not have access to merit forms to remove the primary assignment points. After the instrument is used for a few more years, it will be possible to discern if scholarly and service activity has increased.

A copy of the merit handbook may be obtained by contacting either of the authors at bratcher@nku.edu or schultz@nku.edu.


Appendix A

Original Calendar
(calendar was revised after year one)

May: Committee selected for next year
November: hold organizational meeting
December: chair notify faculty of due dates
February 15: Faculty supervisors submit evaluation of primary assignment to APLS
March 1: Faculty submit activities to SSRC with a copy to supervisor
March 1: APLS send scores of primary assignment to SSRC
March 22: SSRC present merit recommendations to APLS
April 1: APLS submit merit to Provost
April faculty meeting: SSRC present summary
Two weeks following April faculty meeting: committee meets with individuals with questions.


Appendix B

Points for 2004 Evaluation

Activity

Minor

Major

Publications

Abstract

2

5

Review

2

5

Article, non-refereed

6

8

Article, refereed

10

20

Book chapter

10

20

Editor of a book

15

20

Editor of a journal

15

20

Book

40

50

Other publications

Committee decision

Works in progress

Committee decision

Teaching

Teaching college or university class

 

18

Presentations

International/National

10

15

Regional/State/Local

6

10

Campus/Library

2

4

Seminar/Workshop

6

10

Guest lecture in class

2

4

Exhibits

2

6

Grant writing

2

4

Consulting

2

4

Classes taken (semester long)

 

1

Sabbatical

 

5

Attending professional meeting

 

2

Professional Organizations

Offices held in professional organizations

International/National

8

10

Regional/State/Local

6

8

Committee membership

International/National

4

6

Regional/State/Local

4

6

Chair of Committee

International/National

6

8

Regional/State/Local

4

6

Membership in professional organization

 

1

Service

Membership on university committee

2

5

Membership on library committee

2

5

Officer of a university committee

3

6

Officer of a library committee

3

6

Faculty senator

 

4

Other university/library service (School Dazed Booth, President's Golf Outing)

1

4

Community service (serving on a board, judge, etc.)

1

4

Mentoring an individual

1

3

Honors

5

10


Appendix C

Merit Review Process

1. Assign points to listed activities
2. Record points for primary assignment
3. Total points for each faculty member
4. Rank faculty from highest to lowest
5. Form tiers by grouping individuals in clusters
6. The number of tiers can vary per year. It is expected there will be between three and five tiers each year.
7. Units of merit per tier will be:

  • tier 1 = 5 units
  • tier 2 = 4 units
  • tier 3 = 3 units
  • tier 4 = 2 units
  • tier 5 = 1 unit

Appendix D

Points Added or Changed for 2005

Activity

Minor

Major

Publications

Peer reviewer/referee

2

4

Teaching

Course development/first time taught

 

22

Major revision of course + teaching

 

20

One section taught

 

18

Multiple sections taught (2-3)

 

27

Multiple sections taught (4+ )

 

36

Presentations

Poster session

6

8

Moderator

2

4

Other

Funded Research/Projects

Primary writer

10

20

Adjunct writer

5

10

Submitted, not funded/primary

4

8

Submitted, not funded/adjunct

2

4

Grants awarded by an external funder will be assigned major points, grants awarded by an internal funder will be assigned minor points

Professional Development Activities

1

2

Service

Ongoing service (Board member, etc.)

3

6


References

Academic Status for Librarians in ARL Libraries. 1992. SPEC Kit 182. Washington, D.C.: Association of Research Libraries Office of Management Services.

Performance Appraisal. 1979. SPEC Kit 53. Washington, D.C.: Association of Research Libraries Office of Management Studies.

Additional Resources

Downs, Robert B. “Status of Academic Librarians in Retrospect.” In The Case for Faculty Status for Academic Librarians, edited by Lewis C. Branscomb. Chicago: American Library Association, 1970.

Various documents from the following institutions:

University of New Mexico Libraries
University of Cincinnati Libraries
University of Kentucky Libraries
University of Louisville Libraries
Morehead State University Libraries
Eastern Kentucky University Libraries
Xavier University Libraries
Northern Kentucky University College of Business
Northern Kentucky University Dept. of Literature and Languages
Northern Kentucky University Dept. of Physics and Geology


Membership Software Powered by YourMembership  ::  Legal