Public Domain books inacessable at Hathi Trust

jgray · 05-05-2017, 08:37 PM

If this has been discussed before, feel free to delete this post. I did a search of the forums, but didn't find anything.

I don't know what type of deal that Hathi Trust and the associated universities have with Google, but I find it very annoying that books that are clearly labeled "Public Domain" cannot be downloaded without requiring a login from a specific list of universities. Sure, you can view the book online and download one page at a time, but I don't call that proper access for public domain material.

After quite a bit of searching for a downloadable copy of a particular book, I stumbled across something called "Hathi Download Helper".

https://sourceforge.net/projects/hathidownloadhelper/

This program automates the tedious process of downloading one page at a time. It them assembles everything into one PDF for you.

Once installed, set your defaults under the "Options" menu. What works best is to set the download option to "1 pdf per page, searchable text". Also set the "Destination folder" and check "create pdf book after download".

To download a book, paste the URL for that book into the appropriate box in the program and click "Get book info". Next, click "Start download".

Due to the Hathi web server throttling, the program can only download a few pages at a time, then wait five minutes. Once all pages are downloaded, the program will assemble them into a single PDF, with the OCR'ed text included.

What started me on this journey was my mother's cookbook, "Rumford Complete Cook Book", 1950 edition. My sister has what remains of my mother's cookbook, but the covers and several pages are missing. The cookbook saw quite a bit of use over the years.

I did find the 1947 edition online and downloadable, but the recipes were somewhat different from my mother's copy. I was glad that I found the 1950 edition, but then was very annoyed that I could not download this public domain book. I wanted to give a copy to my sister for use on her tablet, and spare the well-used original. Finding the download helper program solved the problem.

When you view a book at Hathi Trust with their online viewer, you will see the typical "Scanned by Google" watermark on every page. It gets worse. On the downloaded PDF pages, Hathi adds another watermark on every page. Well, there are programs that will remove these watermarks, which interfere with the auto-crop feature in many PDF reader programs.

Since the Rumford cookbook was clearly listed as public domain, I removed the watermarks, then spent a few hours creating a table of contents, listing every recipe and other pertinent pages. If you want a copy, you can find it on my web page: www.zianet.com/jgray/

Joe

Cinisajoy · 05-05-2017, 09:51 PM

Joe,
Have you looked on Project Gutenberg for public domain books?
Brb.

jgray · 05-05-2017, 09:57 PM

Yes, I looked everywhere. I was looking for a particular book, which only Hathi Trust had. They seem to have quite a few PD books that were scanned by Google, but don't allow downloading any of them as a complete PDF. Thus, the need for the downloader program I found.

Cinisajoy · 05-05-2017, 10:00 PM

I have the 1946 edition.

Cinisajoy · 05-06-2017, 12:01 AM

<snip> the book looks really good.

Cinisajoy · 05-06-2017, 10:33 AM

<snip>

I wish my copy had been the 1950 edition.

issybird · 05-06-2017, 04:23 PM

Thanks for the link to the tool. And the cookbook.

Hathi's got some WWI memoirs unavailable at Gutenberg.

GrannyGrump · 05-08-2017, 04:22 AM

@jgray -- I have several times downloaded full-length books from Hathitrust, one slow-stepping page at a time, oy veh! Thanks much, wonderful to add another tool to the utility belt.

One thing I find puzzling about the site, is that sometimes identical offerings from different "sources" have different download restriction. An invented example would be:
Huckleberry Finn, original from New York Public Library --- only downloadable with "partner (institution) logon"
Huckleberry Finn, same edition, original from Harvard University -- freely downloadable by anyone.

But download restrictions on different books from the same originating source are not consistent , so it is really mind-bending to figure out who sets the accessability.
I have a theory that a committee of gnomes and goblins has midnight parties, and they throw darts at a list of titles while blind-folded. Bulls-eyes are freely accessible, misses are restricted.

jgray · 05-08-2017, 06:55 PM

You're quite welcome, Granny. I wasn't looking forward to downloading that 200+ page cookbook a page-at-a-time. I lucked out in finding this tool.

On a related note, I'm here to post about some experiments with PDF books whose pages have a color background (from age) and converting to B&W. I'll post that in the PDF section.

crich70 · 05-08-2017, 08:13 PM

I've noticed the problem myself. Some of the books listed over at the "Online Books Page" seem to have links at the Haiti Trust and no where else.

05-05-2017, 08:37 PM	#1
jgray Fanatic Posts: 554 Karma: 2928497 Join Date: Mar 2008 Device: Clara 2E & Sage	Public Domain books inacessable at Hathi Trust If this has been discussed before, feel free to delete this post. I did a search of the forums, but didn't find anything. I don't know what type of deal that Hathi Trust and the associated universities have with Google, but I find it very annoying that books that are clearly labeled "Public Domain" cannot be downloaded without requiring a login from a specific list of universities. Sure, you can view the book online and download one page at a time, but I don't call that proper access for public domain material. After quite a bit of searching for a downloadable copy of a particular book, I stumbled across something called "Hathi Download Helper". https://sourceforge.net/projects/hathidownloadhelper/ This program automates the tedious process of downloading one page at a time. It them assembles everything into one PDF for you. Once installed, set your defaults under the "Options" menu. What works best is to set the download option to "1 pdf per page, searchable text". Also set the "Destination folder" and check "create pdf book after download". To download a book, paste the URL for that book into the appropriate box in the program and click "Get book info". Next, click "Start download". Due to the Hathi web server throttling, the program can only download a few pages at a time, then wait five minutes. Once all pages are downloaded, the program will assemble them into a single PDF, with the OCR'ed text included. What started me on this journey was my mother's cookbook, "Rumford Complete Cook Book", 1950 edition. My sister has what remains of my mother's cookbook, but the covers and several pages are missing. The cookbook saw quite a bit of use over the years. I did find the 1947 edition online and downloadable, but the recipes were somewhat different from my mother's copy. I was glad that I found the 1950 edition, but then was very annoyed that I could not download this public domain book. I wanted to give a copy to my sister for use on her tablet, and spare the well-used original. Finding the download helper program solved the problem. When you view a book at Hathi Trust with their online viewer, you will see the typical "Scanned by Google" watermark on every page. It gets worse. On the downloaded PDF pages, Hathi adds another watermark on every page. Well, there are programs that will remove these watermarks, which interfere with the auto-crop feature in many PDF reader programs. Since the Rumford cookbook was clearly listed as public domain, I removed the watermarks, then spent a few hours creating a table of contents, listing every recipe and other pertinent pages. If you want a copy, you can find it on my web page: www.zianet.com/jgray/ Joe Last edited by issybird; 05-06-2017 at 04:16 PM. Reason: Post restored.

05-06-2017, 12:01 AM	#5
Cinisajoy Just a Yellow Smiley. Posts: 19,161 Karma: 83862859 Join Date: Jul 2015 Location: Texas Device: K4, K5, fire, kobo, galaxy	<snip> the book looks really good. Last edited by issybird; 05-06-2017 at 04:17 PM.

05-06-2017, 10:33 AM	#6
Cinisajoy Just a Yellow Smiley. Posts: 19,161 Karma: 83862859 Join Date: Jul 2015 Location: Texas Device: K4, K5, fire, kobo, galaxy	<snip> I wish my copy had been the 1950 edition. Last edited by issybird; 05-06-2017 at 04:20 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
A to Z of Great Public Domain Books	ficbot	Lounge	24	05-23-2018 10:59 PM
Public Domain Books at Amazon	pdurrant	Amazon Kindle	0	05-01-2010 04:16 PM
Public Domain Day and no PD books until 2026	AprilHare	News	3	01-31-2009 04:54 AM
iPhone Public Domain Books for 99 cents	daffy4u	Apple Devices	64	07-21-2008 07:35 PM

05-05-2017, 09:51 PM	#2
Cinisajoy Just a Yellow Smiley. Posts: 19,161 Karma: 83862859 Join Date: Jul 2015 Location: Texas Device: K4, K5, fire, kobo, galaxy	Joe, Have you looked on Project Gutenberg for public domain books? Brb.

05-05-2017, 09:57 PM	#3
jgray Fanatic Posts: 554 Karma: 2928497 Join Date: Mar 2008 Device: Clara 2E & Sage	Yes, I looked everywhere. I was looking for a particular book, which only Hathi Trust had. They seem to have quite a few PD books that were scanned by Google, but don't allow downloading any of them as a complete PDF. Thus, the need for the downloader program I found.

05-05-2017, 10:00 PM	#4
Cinisajoy Just a Yellow Smiley. Posts: 19,161 Karma: 83862859 Join Date: Jul 2015 Location: Texas Device: K4, K5, fire, kobo, galaxy	I have the 1946 edition.

05-06-2017, 04:23 PM	#7
issybird o saeclum infacetum Posts: 21,234 Karma: 234602041 Join Date: Oct 2010 Location: New England Device: Mini, H2O, Glo HD, Aura One, PW4, PW5	Thanks for the link to the tool. And the cookbook. Hathi's got some WWI memoirs unavailable at Gutenberg.

05-08-2017, 04:22 AM	#8
GrannyGrump Obsessively Dedicated... Posts: 3,213 Karma: 34984928 Join Date: May 2011 Location: JAPAN (US expatriate) Device: Sony PRS-T2, ADE on PC	@jgray -- I have several times downloaded full-length books from Hathitrust, one slow-stepping page at a time, oy veh! Thanks much, wonderful to add another tool to the utility belt. One thing I find puzzling about the site, is that sometimes identical offerings from different "sources" have different download restriction. An invented example would be: Huckleberry Finn, original from New York Public Library --- only downloadable with "partner (institution) logon" Huckleberry Finn, same edition, original from Harvard University -- freely downloadable by anyone. But download restrictions on different books from the same originating source are not consistent , so it is really mind-bending to figure out who sets the accessability. I have a theory that a committee of gnomes and goblins has midnight parties, and they throw darts at a list of titles while blind-folded. Bulls-eyes are freely accessible, misses are restricted.

05-08-2017, 06:55 PM	#9
jgray Fanatic Posts: 554 Karma: 2928497 Join Date: Mar 2008 Device: Clara 2E & Sage	You're quite welcome, Granny. I wasn't looking forward to downloading that 200+ page cookbook a page-at-a-time. I lucked out in finding this tool. On a related note, I'm here to post about some experiments with PDF books whose pages have a color background (from age) and converting to B&W. I'll post that in the PDF section.

05-08-2017, 08:13 PM	#10
crich70 Grand Sorcerer Posts: 11,310 Karma: 43993832 Join Date: Feb 2010 Location: Monroe Wisconsin Device: K3, Kindle Paperwhite, Calibre, and Mobipocket for Pc (netbook)	I've noticed the problem myself. Some of the books listed over at the "Online Books Page" seem to have links at the Haiti Trust and no where else.

Advert

Advert