LOCKSS

LOCKSS
Developer(s)Stanford University
Stable release
1.78.3[1] / 1 August 2024; 4 months ago (1 August 2024)
Repository
LicenseBSD style
Websitewww.lockss.org

The LOCKSS ("Lots of Copies Keep Stuff Safe") project, under the auspices of Stanford University, is a peer-to-peer network that develops and supports an open source system allowing libraries to collect, preserve and provide their readers with access to material published on the Web. Its main goal is digital preservation.

The system attempts to replicate the way libraries do this for material published on paper. It was originally designed for scholarly journals,[2] but is now also used for a range of other materials. Examples include the SOLINET project to preserve theses and dissertations at eight universities,[3] US government documents,[4] and the MetaArchive Cooperative program preserving at-risk digital archival collections, including Electronic Theses and Dissertations (ETDs), newspapers, photograph collections, and audio-visual collections.[5][6]

A similar project called CLOCKSS (Controlled LOCKSS) "is a tax-exempt, 501(c)(3), not-for-profit organization, governed by a Board of Directors made up of librarians and publishers."[7] CLOCKSS runs on LOCKSS technology.[8]

Problem

Traditionally, academic libraries have retained issues of scholarly journals, either individually or collaboratively, providing their readers access to the content received even after the publisher has ceased or the subscription has been canceled.[9] In the digital age, libraries often subscribe to journals that are only available digitally over the Internet. Although convenient for patron access, the model for digital subscriptions does not allow the libraries to retain a copy of the journal. If the publisher ceases to publish, or the library cancels the subscription, or if the publisher's website is down for the day, the content that has been paid for is no longer available.

Methods

The LOCKSS system allows a library, with permission from the publisher, to collect, preserve and disseminate to its patrons a copy of the materials to which it has subscribed as well as open access material (perhaps published under a Creative Commons license). Each library's system collects a copy using a specialized web crawler that verifies that the publisher has granted suitable permission. The system is format-agnostic, collecting whatever formats the publisher delivers via HTTP. Libraries which have collected the same material cooperate in a peer-to-peer network to ensure its preservation. Peers in the network vote on cryptographic hash functions of preserved content and a nonce; a peer that is outvoted regards its copy as damaged and repairs it from the publisher or other peers.[10][11]

The LOCKSS license used by most publishers allows a library's readers access to its own copy, but does not allow similar access to other libraries or unaffiliated readers; the system does not support file sharing. On request, a library may supply another library with content to effect a repair, but only if the requesting library proved that in the past that it had a good copy by voting with the majority. If the reader's browser no longer supports the format in which the copy was collected, a format migration process can convert it to a current format.[12] These limits on the use that may be made of preserved copies of copyright material have been effective in persuading copyright owners to grant the necessary permission.[13]

The LOCKSS approach of selective collection with permission from the publisher, distributed storage, and restricted dissemination contrasts with, for example, the Internet Archive's approach of omnivorous collection without permission from the publisher, centralized storage, and unrestricted dissemination. The LOCKSS system is far smaller, but it can preserve subscription materials to which the Internet Archive has no access.

Since each library administers its own LOCKSS peer and maintains its own copy of preserved material, and since there are libraries doing so worldwide (see the list of participating libraries below), the system provides a much higher degree of replication than is usual in a fault-tolerant system. The voting process makes use of this high degree of replication to eliminate the need for backups to off-line media, and to provide robust defenses against attacks aimed at corrupting preserved content.[14]

Importance

In addition to preserving access, libraries have traditionally made it difficult to rewrite or suppress printed material. The existence of an indeterminate but large number of identical copies on a somewhat tamper-resistant medium under many independent administrations meant that attempts to alter or remove all copies of a published work would likely both fail and be detected. Web publishing, based on a single copy under a single administration, provides none of these safeguards against subversion. Web publishing is, therefore, an amenable tool for rewriting history. By preserving many copies under diverse administration, by automatically auditing the copies at intervals against each other (and, in the future, against the publisher's copy), and by alerting libraries when changes are detected, the LOCKSS system attempts to restore many of these safeguards in the now digital world of publication.

Implementation

Prior to implementing a LOCKSS system, some questions need to be considered carefully in order to make sure the content is verified, evaluated, and auditable by users. The user must ask questions such as, "What are your procedures?", "What are your methods?", "How is this system evaluated?", and "What is your disaster preparedness program?". These questions will enable the user to evaluate the system, create a successful maintenance plan for their materials, and enable the system to be reinforced by a carefully evaluated support structure.

The source code for the entire LOCKSS system carries BSD-style open-source licenses and is available from GitHub.[15] LOCKSS is a trademark of Stanford University.

See also

References

  1. ^ "Release 1.78.3". 1 August 2024. Retrieved 22 August 2024.
  2. ^ David S. H. Rosenthal; Vicky Reich (June 18, 2000). Permanent Web Publishing (PDF). Proceedings of FREENIX Track: 2000 USENIX Annual Technical Conference. Retrieved 2008-01-19.
  3. ^ "ASERL and LOCKSS to Preserve e-Theses & Dissertations" (Press release). SOLINET. July 11, 2005. Retrieved 2008-01-19.
  4. ^ Jacobs, James. "LOCKSS-USDOCS". home page. LOCKSS. Retrieved 23 February 2012.[permanent dead link‍]
  5. ^ "The MetaArchive Cooperative". Home page. Retrieved 2008-01-19.
  6. ^ Maniatis, Petros; Rosenthal, David S. H.; Roussopoulos, Mema; Baker, Mary; Giuli, TJ; Muliadi, Yanto (2003). "Preserving peer replicas by rate-limited sampled voting". Proceedings of the nineteenth ACM symposium on Operating systems principles - SOSP '03 (PDF). p. 44. arXiv:cs/0303026. doi:10.1145/945445.945451. ISBN 978-1581137576. S2CID 215753435. Free PDF download.
  7. ^ CLOCKSS.org (2015), CLOCKSS.org.
  8. ^ CLOCKSS.org (2008-02-14), What's the difference between LOCKSS and CLOCKSS? (PDF), retrieved 2015-11-21.
  9. ^ "Preservation Principles - LOCKSS". Archived from the original on 2018-12-01. Retrieved 2013-03-07.
  10. ^ Petros Maniatas; Mema Roussopoulos; TJ Giuli; David S. H. Rosenthal; Mary Baker; Yanto Muliadi (October 19, 2003). "Preserving Peer Replicas By Rate-Limited Sampled Voting" (PDF). ACM Symposium on Operating Systems Principles. Archived from the original (PDF) on January 28, 2005. Retrieved May 2, 2022.
  11. ^ T.J. Giuli; Petros Maniatis; Mary Baker; David S. H. Rosenthal; Mema Roussopoulos (November 27, 2004). "Attrition Defenses for a Peer-to-Peer Digital Preservation System". arXiv:cs.CR/0405111.
  12. ^ David S. H. Rosenthal; Thomas Lipkis; Thomas S. Robertson; Seth Morabito (January 2005). "Transparent Format Migration of Preserved Web Content". D-Lib Magazine. 11 (1). arXiv:cs/0411077. doi:10.1045/january2005-rosenthal. S2CID 474. Retrieved 2008-01-19.
  13. ^ "Publishers and Titles". LOCKSS. Archived from the original on 2008-01-08. Retrieved 2008-01-19.
  14. ^ David S. H. Rosenthal & Daniel L. Vargas (September 11, 2012). "LOCKSS Boxes in the Cloud" (PDF). Retrieved October 11, 2013.
  15. ^ "LOCKSS (Lots of Copies Keep Stuff Safe)". Project web site. GitHub. Retrieved September 12, 2017.
  16. ^ Rosenthal, David S. H.; Reich, Vicky (2000-06-18). "Permanent web publishing" (PDF). Proceedings of the Annual Conference on USENIX Annual Technical Conference. ATEC '00. USA: USENIX Association: 40. Archived from the original on 18 June 2000. The project was to a large extent inspired by Danny Hillis & Stewart Brand's Millennium Clock project[clock].

Further reading