On behalf of the Ontario Council of University Libraries (OCUL), the Center for Research Libraries conducted a preservation audit of the Scholars Portal repository as a repository of journal content. The audit took place between January 15, 2012 and July 15, 2012. Based on the audit findings, CRL hereby certifies that the Scholars Portal journal repository is a trustworthy digital repository for e-journal content.
While Scholars Portal maintains a wide variety of types of digital content, including still and moving images, and audio, geospatial and statistical information, the CRL audit focused on the repository’s ability to manage licensed electronic journal titles, issues and articles. The audit’s purpose was to determine whether that content is preserved by Scholars Portal in a way that meets the expectations of the OCUL community and conforms to accepted criteria for trusted digital repositories.
CRL’s analysis was guided by the criteria included in the Trustworthy Repositories Audit and Certification (TRAC) checklist, and other metrics developed on the basis of CRL analyses of digital repositories. CRL’s assessment involved a review of information provided by Scholars Portal, documentation and information available independently from open and third-party sources, and a site visit by CRL audit personnel, Marie Waltz and James Jacobs, to the University of Toronto on April 17-18, 2012.
CRL’s Certification Advisory Panel (CAP) reviewed all of the evidence provided. On the basis of this evidence, the Panel concluded that the practices and services described in Scholars Portal’s public communications and published documentation are generally sound, and appropriate to both the journal content being archived and the needs of the stakeholder community. Moreover, the Panel expects that in the future Scholars Portal will continue to be able to deliver journal content that is understandable and usable by its specified user communities.
CRL assessed the Scholars Portal e-journals database repository on each of the three categories of criteria specified in TRAC and has assigned a level of certification for each. The numeric rating (below) is based on a scale of 1 through 5, with 5 being the highest level, and 1 being the minimum certifiable level.
TRAC Category |
Scholars |
Optimum |
Organizational Infrastructure |
5 |
5 |
Digital Object Management |
4 |
5 |
Technologies, Technical Infrastructure, Security |
4 |
5 |
TOTAL |
13 |
15 |
The basis of these ratings is found in Section B. Detailed Audit Findings.
Scholars Portal
Scholars Portal (http://scholarsportal.info [4]) was created in 2002, as a repository for digital content licensed, purchased, and otherwise acquired by the 21 Canadian libraries represented by the Ontario Council of University Libraries (OCUL) consortium. Scholars Portal provides a common technical infrastructure for delivering digital content and services to support research, teaching, and learning within the Ontario higher education community. Scholars Portal was initially created as a local hosting solution for licensed content. It has since evolved to include digital preservation, archiving and repository services.
The Center for Research Libraries
The Center for Research Libraries (CRL) (www.crl.edu [5]) is an international consortium of university, college, and independent research libraries. CRL supports advanced research and learning in the humanities, sciences, and social sciences by ensuring the survival and accessibility of source materials vital to those disciplines. In order to enable its community to accelerate the shift to electronic-only resources in a careful and responsible manner, CRL both preserves and maintains shared physical collections of materials and evaluates digital repositories of interest to its community.
The members of the CRL Certification Advisory Panel for the Scholars Portal audit were:
Perry Willett (Chair) |
Barbara McDonald |
Winston Atkins |
William R. Parod, Jr. |
Pascal Calarco |
Anne Pottier |
Catherine Davidson |
Oya Y. Rieger |
This assessment was undertaken to determine whether or not the Scholars Portal e-journals archive meets the commitments the repository’s management has made to its stakeholders with regard to the long-term preservation of e-journals for the academic research community and whether the repository complies with established criteria for trusted digital repositories. The assessment included a site visit, a review of information independently gathered by CRL from published and unpublished sources, and a review of documents and documentation provided by Scholars Portal.
CRL conducted its audit with reference to:
The primary metrics used by CRL in such assessments are contained in the Trustworthy Repositories Audit and Certification (TRAC) checklist and a newer incarnation of this checklist, the Audit and Certification of Trustworthy Digital Repositories (TDR) checklist. TRAC was developed by a joint task force created by the Research Libraries Group (RLG) and the National Archives and Records Administration in 2003 to provide criteria for use in identifying digital repositories capable of reliably storing, migrating, and providing long-term access to digital collections. TRAC represents best current practices and thinking about the organizational and technical infrastructure required for a repository to be considered trustworthy and thus worthy of investment by the research and research library communities. The approved ISO standard, TDR (ISO 16363), was also used in this audit. Because there is currently no ISO-approved mechanism for accrediting certifying bodies for the TDR standard, CRL’s certification is to TRAC criteria.
CRL assessed the Scholars Portal e-journals database repository on each of the three categories of criteria specified in TRAC and has assigned a level of certification for each. The numeric rating (below) is based on a scale of 1 through 5, with 5 being the highest level, and 1 being the minimum certifiable level. (The minimal certification rating of 1 is assigned in instances where a repository has inconsistencies or deficiencies in areas that might lead to minor defects of a systemic or pervasive nature, but where no major flaws are evident.)
[1]TRAC - /attachments/pages/trac_0.pdf [7]
[2] TDR - http://public.ccsds.org/publications/archive/652x0m1.pdf [8]
[3]OAIS - http://public.ccsds.org/publications/archive/650x0m2.pdf [9]
CRL assessed the Scholars Portal e-journals database repository on each of the three categories of criteria specified in TRAC and has assigned a level of certification for each. The numeric rating (below) is based on a scale of 1 through 5, with 5 being the highest level, and 1 being the minimum certifiable level. (The minimal certification rating of 1 is assigned in instances where a repository has inconsistencies or deficiencies in areas that might lead to minor defects of a systemic or pervasive nature, but where no major flaws are evident.)
TRAC Category |
Scholars |
Optimum |
Organizational Infrastructure |
5 |
5 |
Digital Object Management |
4 |
5 |
Technologies, Technical Infrastructure, Security |
4 |
5 |
TOTAL |
13 |
15 |
On the basis of the audit, CRL found the Scholars Portal e-journals archive to be in compliance with the TRAC criteria. Scholars Portal has met all TRAC requirements at this time. CRL found that Scholars Portal’s management of journal content complied with the 84 applicable individual criteria in the three primary areas assessed by TRAC: administration, object management and technical systems. Notable positive findings of the audit are as follows.
TRAC Section A: Organizational Infrastructure
Succession Planning [A1.2] Scholars Portal has an appropriate, formal succession plan in place for its e-journals archive. One of the important features of this plan is that OCUL institutions have collective ownership and rights to access and preserve Scholar’s Portal content in perpetuity. Collective ownership ensures that should the Scholar’s Portal repository cease to operate or OCUL substantially change its scope, the content may be moved and preserved elsewhere without restriction. It also means that no further permissions or licenses must be sought before access to and preservation of the content can be provided.
Procedural Accountability and Policy Framework (Documentation) [A3] Scholars Portal has created a detailed, transparent framework of policies and procedures that are fully consistent with the repository’s stated preservation strategies. It has made this material publicly available on the web at http://spotdocs.scholarsportal.info/display/OAIS/Document+Checklist [10]. The documentation clearly discloses the benefits and preservation services of the repository in a way that can serve as a model for other repositories. The disclosures also demonstrate the commitment of Scholars Portal management to transparency and to accountability to its stakeholder community.
TRAC Section B: Digital Object Management
Preservation Planning [B.3] Scholars Portal has explained and documented in detail its preservation planning in three key documents: Preservation Strategic Plan, Preservation Implementation Plan, and Preservation Action Plan - Journals, and in its on-going Risk Analysis and Management Strategies document. Scholars Portal is committed to reviewing its preservation policies every two years, as indicated in its Review Cycle for Documentation Policy.
Monitoring Integrity of Archival Objects [B4.4] Scholars Portal checks all existing Archive Ingest Packages (AIPs) for fixity. At the time of ingest three checksums are created for each digital object. Checksum tests are recorded in the object’s metadata using PREMIS event entities and are publicly reported in a “TDR Status” view for each journal article.
Information Management [B.5] Scholars Portal has a robust and useful user interface for information discovery that is based on well managed, rich descriptive information stored in the e-journals archive. Members of the repository staff regularly seek feedback on user experiences.
Access Management [B.6] Scholars Portal access logs are regularly monitored for errors, unauthorized or unusual activity, and for the successful completion of requests. Staff members respond to problems by creating problem reports in the JIRA issue tracking system.
TRAC Section C: Technical Infrastructure, Security
Technical Infrastructure and Risk Management [C.1] Scholars Portal maintains its own hardware data center in a secure facility with closely monitored and controlled access. The center is well designed and maintained, well-organized, and appropriately cooled and climate-controlled.
Backups and Disaster Recovery [C 1.2] The repository has adequate backups of its data to preserve the contents of the archive and an adequate Disaster Recovery Plan to deploy those backups to bring all archived content back online after a catastrophic event within a timeframe acceptable to the repository’s designated community.
It is important to note that Scholars Portal does not, however, have an online “mirror site.” It operates out of a single data center in Toronto. During the audit, the CRL Certification Advisory Panel expressed the concern that lack of a mirror site means that, in the event of a disaster Scholars Portal could take weeks or longer to restore the e-journal archive -- resulting in a significant interruption in service. Scholars Portal was able to confirm that its designated user community understands and accepts this situation.
The OCUL Executive Committee, representing that community, provided CRL the following statement:
In the view of this committee, Scholars Portal’s current recovery planning and service levels fulfill the needs of the OCUL community. While we recognize that an unplanned interruption of service would be an inconvenience to OCUL’s user communities, the long-term preservation of the locally loaded materials is of highest importance and as long as the material can be recovered and its integrity and authenticity re-established, the needs of the Designated Community are being met.
Scholars Portal acknowledges that implementation of an online mirror site would provide an additional layer of security and ensure continuous service in the event of a disaster and that mirroring would provide online redundancy of archival storage, data management, and dissemination systems. Consequently, at the time of this report Scholars Portal and its administrators, the Ontario Council of University Libraries, were endeavoring to engage another entity to take on the role of a mirror site and CRL encourages them to continue this activity.
Disaster Recovery Plan [C3.4] This TRAC metric requires that a repository have suitable written disaster preparedness and recovery plan(s), including at least one off-site backup of all preserved information together with an off-site copy of the recovery plan(s). Scholars Portal has a detailed, written Disaster Recovery plan based on working, tested procedures. In addition they have contracted with an outside vendor to house a copy of their backup. Copies of the disaster plan are also maintained offsite, should the University of Toronto site itself become inaccessible.
The TRAC document notes that “. . . attaining trusted status is not a one-time accomplishment—achieved and forgotten. To retain trusted status, a repository will need to undertake a regular cycle of audit and/or certification.” To that end CRL and Scholars Portal have agreed that ongoing certification is contingent upon Scholars Portal making the following disclosures every three years:
Certification is also contingent upon Scholars Portal’s agreement to a periodic, systematic sampling and inspection of the repository’s archived content by CRL, or by a third party designated by CRL, using either a manual or automated process as determined by mutual agreement between CRL and Scholars Portal.
Links
[1] https://www.crl.edu/sites/default/files/reports/ScholarsPortal_Report_2013_%C6%92.pdf
[2] https://www.crl.edu/facets/archiving-and-preservation
[3] https://www.crl.edu/reports
[4] http://spotdocs.scholarsportal.info/display/sp/home
[5] https://www.crl.edu/
[6] http://public.ccsds.org/publications/archive/650x0m2.pdf
[7] https://www.crl.edu/attachments/pages/trac_0.pdf
[8] http://public.ccsds.org/publications/archive/652x0m1.pdf
[9] http://public.ccsds.org/publications/archive/650x0b1.pdf
[10] http://spotdocs.scholarsportal.info/display/OAIS/Document+Checklist