4. Plan of Work--Bibliographic Analysis


4.3.3 Scholarly Evaluation and Ranking

Following compilation of the state bibliographies, each library will engage a scholarly review panel of
three to five individuals to review and rank the titles in the bibliography for preservation priority
according to the rating scheme described below. (See Section 5, Preservation Profiles and Project Staff
for the list of scholarly reviewers from each state.) Reviewers were selected for their knowledge of
agriculture and/or state history and chosen to reflect different backgrounds and points of view. Each
reviewer will be paid an honorarium ($450 per person in most cases; these costs are included in the per
volume cost for each state) for his or her services as a consultant to the project and asked to deliver the
completed product within six weeks. Based on the Phase 1 experience, it is estimated that each reviewer
will spend approximately eight hours reviewing the list, usually over a period of several weeks.

Prior to distributing the bibliography for review, each project library will host a meeting for the reviewers
to discuss the project as a whole, the scope of the bibliography and the compilation process, and the
rating scheme and scholarly review process. This meeting will help insure that reviewers are apply the
rating scheme in a similar manner and that questions are addressed up front.

Monograph citations will be separated into approximately ten subject categories to help structure the
review process and make it easier for the reviewers to consider each title within the context of the subject
matter and the time period of the publication. Likewise, serials titles will be arranged in categories
depending upon the type of publication ( e.g. serials about state agriculture, land grant university
publications, etc.). Reviewers will also receive a separate list of the titles that have already been preserved
in an acceptable preservation format. (See Section 4.4.1 Searching and Identification below.)

Following the meeting, the reviewers will be asked to determine the relative importance of each title for
historical research in the humanities, as compared to other titles within the same list and according to the
following scheme:

 First Priority: Very important historical title, of critical importance to preserve in this grant
 project.

 Second Priority: Important title definitely worth preserving in this grant project, funds permitting.

 Third Priority: Worth preserving at some time, but of a lower priority.

 Fourth Priority: Not worth preserving.

 In addition to the scholarly reviewers, the institutional project managers, or another appropriate person in
 the institution, will also evaluate the list from the point of view of an agriculture bibliographer. A review
 by a practicing librarian will contribute a broad perspective on how the materials are used by students and
 scholars in many disciplines and a sense of the relative scarcity of the publications.

 Following the rating process, the results for each title will be averaged and ranges assigned in order to sort
 the titles into logical and manageable priority groups. For example, in New York, titles that fell into the
 1.0 1.5 range became first priority titles; titles that fell into the 1.6 to 2.0 range were ranked second
 priority for the project. The experience of the original New York State project and the Phase 1 project
 was that there was remarkable similarity in the way that historians viewed the literature. A random
 sampling of the ratings showed that the four reviewers rated each title either the same or one rating apart
 more than 90% of the time.