"Digital Books wear out faster than Physical Books" (Internet Archive)

[retiredbiker]

Ten years ago we moved to a new house. It had a nice dry basement with no hint of any prior flooding events. So we stored dozens of cartons of paper books down there, pending unpacking and shelving. There was an ice storm and some storm drains in the area became blocked. Ten inches of water in the basement for a few hours. A "once in a generation event" said the local paper.

The lifetime of about a thousand paper books: two or three minutes. Right then my ebook collection began to grow like Topsy.

Internet Archive aims to gather and save for posterity all the knowledge there is. So why does it base itself in a time and place where that is impossible with all the major publishers and nut cases breathing down their neck? And their attempts to get around that pressure like "one hour loans" are pathetic.

And the quality control, at least in the small bit I use is simply terrible. Things scanned in China, twisted, misaligned, pages missing, no checks of any kind. Automated OCR and text formats, including epub, that are simply unreadable. How is this saving information for the future? It is like intentionally screwing it up to provide job security for some future generation of digital archaeologists.

In many ways, the shadow libraries do a better job.

[ZodWallop]

Originally Posted by haertig

In the very first paragraph of the article:

Originally Posted by cfrizz

The whole article did it with the message being paper is better because you can pass your books on to others to read, you don't have to worry about the paper book becoming obsolete, blah, blah blah!

I didn't get that at all. I think you just took the headline and ran with it. Why would Internet Archive be championing paper books?

Originally Posted by tomsem

The point of the article is to justify the existence of Internet Archive, and to point out that it requires ongoing work to maintain.

Yes, exactly.

[ZodWallop]

Originally Posted by retiredbiker

Ten years ago we moved to a new house. It had a nice dry basement with no hint of any prior flooding events. So we stored dozens of cartons of paper books down there, pending unpacking and shelving. There was an ice storm and some storm drains in the area became blocked. Ten inches of water in the basement for a few hours. A "once in a generation event" said the local paper.

The lifetime of about a thousand paper books: two or three minutes...

That was me when a water pipe burst in the garage roof. Thirty+ years of comic collecting...

[Aleron Ives]

Originally Posted by Deskisamess

In a readable condition? Have you seen the quality of many current paper books?

Yes, I should have been more careful and specified paper, rather than paper books. I highly doubt anyone would have been able to read the Dead Sea Scrolls if they had been stored on a hard drive, rather than paper.

[ZodWallop]

Originally Posted by Aleron Ives

I highly doubt anyone would have been able to read the Dead Sea Scrolls if they had been stored on a hard drive, rather than paper.

Yeah, something that old, they likely would have been on those 8" floppies and who can handle those?

[haertig]

Originally Posted by Aleron Ives

I highly doubt anyone would have been able to read the Dead Sea Scrolls if they had been stored on a hard drive, rather than paper.

The Dead Sea Scrolls would be interesting to trace back to their origin. But the hard drive that somebody stored them on? Even more interesting!

[MGlitch]

I dunno, books from the 90s aren’t really holding up well unless stored in some great conditions, the pages may have gotten a little dried out, the ink has faded, etc.

Meanwhile I’m reasonably sure I can open a text file from a Mac Classic on my iMac M1 despite the decades of time and OS upgrades between the two. And it would look just as it had back in the 90s. Arguably better because the screen I’m reading on is going to be miles away sharper.

[DNSB]

Originally Posted by ZodWallop

Yeah, something that old, they likely would have been on those 8" floppies and who can handle those?

Moi? A week ago, I finished transferring some old corporate records from IBM EBCDIC 8" single sided single density floppies to a USB drive for a corporate customer. I will admit that was the first time I've used those drives in 6 years and I was very happy that my hand written (more like scribbled) notes attached to the drive case were still readable (not to mention the both the Qume drives still worked). Of the ~80 disks, only 2 had non-recoverable read errors.

[meeera]

Originally Posted by retiredbiker

Ten years ago we moved to a new house. It had a nice dry basement with no hint of any prior flooding events. So we stored dozens of cartons of paper books down there, pending unpacking and shelving. There was an ice storm and some storm drains in the area became blocked. Ten inches of water in the basement for a few hours. A "once in a generation event" said the local paper.

I'll never forget the way my heart sank when I saw video of the flooding of the Education/Fine Arts/Architecture library in the Great Perth Hailstorm. 15% of the collection was lost.

https://youtu.be/jSNR6EpwKn4

[Tex2002ans]

On Physical Preservation

These absolutely fantastic videos cover 2 major archives being lost:

• The History Guy: "The National Personnel Records Fire of 1973"
• The History Guy: "History Lost: The Tragedy of the 1890 Census"

Records + physical paper also:

• Take up a ton of space
• Are rotting with the passage of time
  
  • And natural disasters (fires, floods, etc.)

so these also have to constantly be maintained + are getting converted/condensed to other formats.

Like microfilm was used for many decades... but even with that, you have to store it somewhere... and, better archival tech can always come along. See:

• 2020: Internet Archive Blog: "Computerworld Archives: Back From Vintage Microfilm"

where they linked to early microfilm->digital conversions, then current 2020 microfilm->digital conversions.

The quality is so much better with the newer scanning tech.

- - -

Another example is many old newspapers/magazines converted to microfilm, with the originals destroyed (or rotted away).

Like a recent book I've been digitizing referenced a lot of the:

• Richmond Enquirer (1815–1867)

You can already see severe:

• yellowing
• water stains
• torn edges

... and trying to scroll through the Library of Congress's website, it's just awful. You have to:

• Download/View 1 page at a time
  
  • Only available as PDF + "JPEG2000".
• Get awfully, super-compressed images

And you can't easily do text searches or anything... (Not that their OCR is great either...)

Side Note: If you want more details, I wrote a bit about newspaper OCR last month in:

• /r/LibreOffice: "How do I fix formatting issues?"

Yes, the Library of Congress version is "available"—and better than nothing—but it's horrible to actually use + get anything useful out of it!

- - -

Physical Preservation (Old Photos/Films) + Old Tech

Originally Posted by haertig

Similarly, the slides I took on our honeymoon are deteriorating. [...] Besides, I don't even know where my slide projector is. Chances are it would need a new bulb (probably unavailable). And additional carousels to hold the slides. I wonder where in the heck my roll down slide screen is?

Heh, exactly!

I think you'll be very interested in this fantastic documentary:

• Side by Side (2012)

which was Keanu Reeves covering the changeover from film -> digital.

I wrote about it in a sidenote back in 2013:

• 2013: "commercial software for Kindle/mobi from scanned math texts"

But like you+DNSB said with your slide projectors/floppies, even the tech to read these some of these formats is rotting away. (See the documentary for many examples of early digital movies/formats being completely unreadable now.)

- - -

Side Note: And if you love video games + early computer animation/CGI, see:

• 2019: "DF Retro EX: Making Pitfall, Tron & Beyond - An Interview with Bill Kroyer"

where they discuss how movies like Tron + Terminator were done, but those old computers were running proprietary hardware/software/OSes, it was all custom-made stuff... so it's very hard (or impossible) to even recover those originals.

- - -

If you like old films, there are also ones completely lost in time too, like:

• Saved from the Titanic (1912)

which was actually shot a few weeks after the Titanic sunk, with an actual actress who survived the shipwreck (wearing the same exact clothes)!

If you're interested in that type of stuff, check out the:

• Wikipedia.org: "Lost Film"

A report created by Library of Congress film historian and archivist David Pierce claims:

• 75% of original silent-era films have perished.
• 14% of the 10,919 silent films released by major studios exist in their original 35 mm or other formats.
• 11% survive only in full-length foreign versions or film formats of lesser image quality.

Of the American sound films made from 1927 to 1950, an estimated half have been lost.

And every so often, someone appears with some long-lost versions, sitting in someone's private vault somewhere:

• Wikipedia.org: "List of rediscovered films"

(Like lost episodes of "Dr. Who" pop up every few years!)

Physical Preservation (Vinyl)

In 2021, the Internet Archive also posted this amazing tweet showing how they digitize all these vinyl records:

• https://twitter.com/internetarchive/...23512810721284

And this 2019 article on their blog about the Boston Library collection:

• Internet Archive Blog: "How the Internet Archive is Digitizing LPs to Preserve Generations of Audio"

On Updating/Maintaining Formats

Even ebooks need to be maintained... see the fantastic article+speech by Teresa Elsey (an editor at Houghton Mifflin):

• 2016: "When Nothing Ever Goes Out of Print: Maintaining Backlist Ebooks"
• ebookcraft 2019: "Building Ebooks that Last"

Link Rot alone is a serious problem. Every year, a large % of all URLs break:

Hyperlinks

Many of our books have URLs in them. Particularly our adult nonfiction books, which often have endnotes with lots of URLs. And in ebooks, we make these URLs into hyperlinks.

And as we know, URLs can stop working.

The web community has gotten good about talking about this problem, and they call it link rot.

There’s scholarly research on the prevalence of link rot—when URLs stop working—and of reference rot—when the information at a given URL changes from what it was when the author cited it. This study found that more than half of the URLs cited in US Supreme Court decisions suffered from one or the other. Which is not a great thing for the history of American jurisprudence.

[...]

For more, also see my discussion back in:

• 2017: "Citing Wesbites"

On Updating/Maintaining News/Laws + Completely New Interactive/Analysis

This year, the Internet Archive's big push was:

• "Building Democracy's Library"

where they're digitizing tons of government documents, and making them actually usable/searchable. (Right now, a lot of it is buried in physical form somewhere, completely inaccessible to the public.)

Another recent project is that they've been digitizing decades of news:

• 2022: "Library as Laboratory Recap: Opening Television News for Deep Analysis and New Forms of Interactive Search"
  
  • 2022: "Television As Data" (April 13, 2022)
    
  • 2022: "A New Approach To Understanding War Through Television News: Introducing The TV News Visual Explorer & The Belarusian, Russian & Ukrainian TV News Archive"

Now, you're able to:

• Search through closed captions + subtitles
• Find exact clips/minutes of any news over the past ~30 years.
• Translate + do all sorts of data analysis on it.
  
  • For example, seeing topics/trends over time—similar to Google Trends—or seeing how many times a specific person appeared or was interviewed (based on visuals/captions on the screen as well).

This is the kind of stuff the original article was talking about. You have to constantly reprocess the backlog, using a lot of the latest tech, to make the old documents/scans even more findable/searchable/usable.

It's not just some static vault—where you lock things away, and maybe 1 person in a decade might even see the dusty tome—there's constantly new/better ways of accessing this data.

[Quoth]

Originally Posted by DiapDealer

But all prints are made from digital copies these days. So if the future of the written word is to rely upon the personal paper collections of readers (better long term storage), we're doomed. I'll take my chances with digital. The medium all new, and reprinted books will be created from.

Unless we're just talking personal storage, here. In which case I'll still take digital. I'm not a collector.

Digital is fine unless the format can only be opened by a program on OS/2 (worst support for a VM).

Formats and digital storage are not problems. It's DRM that's an issue for ordinary people. There isn't any DRM that can't be defeated by experts even if server is gone. See MS "Plays for sure".

Cheap paper doesn't last. The 100+ year old books don't use the paper called Pulp in 1930s. My 100+ year old books better than may 1960s to 1980s.

If you want physical print then parchment or better still, clay tablets.

[Cactus Chef]

Originally Posted by haertig

This is why the saying goes: "RAID is not a backup system". RAID stands for "Redundant Array of Independent Disks". So if you experience corruption on one disk, then RAID diligently copies that corruption to the other disks in the array, so you end up with "redundant corruption". RAID is (should be!) used to gain better system uptime or speed. It doesn't do diddly squat for backups. RAID systems need to be backed up just the same as non-RAID systems do. I guess IA didn't know that.

If only they had employed a Sun/FreeBSD greybeard. Would've told them to use a ZFS RAID.

[Quoth]

Originally Posted by haertig

This is why the saying goes: "RAID is not a backup system". RAID stands for "Redundant Array of Independent Disks". So if you experience corruption on one disk, then RAID diligently copies that corruption to the other disks in the array, so you end up with "redundant corruption". RAID is (should be!) used to gain better system uptime or speed. It doesn't do diddly squat for backups. RAID systems need to be backed up just the same as non-RAID systems do. I guess IA didn't know that.

RAID, like a Cluster, is simply high availability, not a backup.
And a backup is only local HW/SW failure, not location failure unless there is an off-site copy. Fire, break-ins, subsidence, flood, lightning, truck/car hitting building, bombs, terror, missiles, earthquakes.

Online copies are more about collaboration & sharing than backups.

Mirroring isn't too bad now, but RAID5 rebuild time has got so silly that having backup HDDs offline and copying to a reformatted RAID5 is far faster than a rebuild replacing one 6T drive. The last RAID5 I had was ultra-wide & fast 15000 RPM SCSI drives. Certainly under 5G byte each. HW RAID controller with cache DRAM and battery. But really needed active UPS to avoid serious data loss/corruption on power fail. Prior system was Pentium Pro dual CPU with EISA RAID controller and "hot swap" caddies, only 2G Byte 10,000 RPM SCSI.
Even in late 1990s I got cross with salesman when he pulled out a caddy on our main server as demo. Took hours to rebuid. OTOH the first NT 4.0 cluster (regualar servers with dual external SCSI ports and SCSI extenders as isolator/auto-termination) we had with two sets of mirrored external drives had no issue with pulling power cord out on one server. It did need 4x UPSes!

I gave away my last 8" drive a few years ago, but still have 360K & 1.2M 5.25", both kinds of 3" and 720K and 1.44M 3.5" and one last Mobo with PCI-e and decent CPU with a real floppy port. A USB 3.5" can't read "foreign" formats. Drivers and Emulators for CP/M, DOS, Amstrad PCW and others (not Amiga floppies) on Linux. Some on a much older P4 XP box.

Dumped last PCs with ISA and MFM HDD a few years ago.

[Greg Anos]

Permanence in a "throw away" society? <shrug>

If one cares, technology can be made more durable; but nobody really cares (except for a few oddballs). Think vintage cars, for example.

Yes, digital information needs to be maintained. OTOH, maintenance is usually simple, if one does it in a timely manner. Most of the "horror stories" come from people/organizations who wouldn't bother to do that maintenance until the tech was so old that conversion methods had been scrapped. 8" floppy drive in 2022? Why hadn't they been converted decades ago to 5 1/4" floppies in the 1980's, and those to 3 1/2 (or hard drives) in the 1990's? (and those multiply backed up?) A days work both times.

I keep my data in commonly used, open sourced, formats; and in on-site, and off-site, storages. And for the most part, I use it on off line (no internet access) machines. My data is far more valuable that the machines it's used on, or the storage it's stored on.

It's DRM that kills data, much more so than obsolescence,

[Sarmat89]

For decades, libraries discarded and burned old junk that took up their shelves. Now, there is the 'preservation' craze. It's ridiculous.