Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 03-20-2009, 09:49 PM   #1
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
Sony and google books, anyway to bulk download all free books?

Just wondering that now that sony and goodle offer all their old books collection in epub format, is there a way to bulk download it all. I M A HUGE book collector, so having scans of the originals as Google does is 100% better to me, so i was wondering if there was a way torrent or other to get the free collection other than clicking one by one... i ll grow to an old man before i m done otherwise !


any tips are very very welcomed !
Student1 is offline   Reply With Quote
Old 03-20-2009, 10:15 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,109
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
you could probably use some sort of web scraper to do it.
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 03-20-2009, 10:23 PM   #3
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
You know that is a great idea! I bet pagenest would do it! I was getting them from the sony software but i bet there is a web page were they can be found! Thanks for the tip i m on the trail !
Student1 is offline   Reply With Quote
Old 03-20-2009, 10:42 PM   #4
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
Hmm seems these epub books are only downloadable from the sony software, so a web scraper might not work in this case. Unless there is a web page that host them... but http://books.google.com/ doesn t seem to offer any download option for the epub format...
Student1 is offline   Reply With Quote
Old 03-21-2009, 01:57 AM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,109
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
the google books data api may have calls to access the books, though I'm not sure
kovidgoyal is offline   Reply With Quote
Old 03-21-2009, 03:40 PM   #6
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
It probably would request a password from the site, but that would mean writing an application to authenticate the download and request from the server the files...

Coming back to the web scraper idea, i tried the google book search page and found out all books are there and in PDF format, they are all scanned so i m guessing there must be an easy way to convert from pdf scan to epub scan, much like pdf read does for lrf. So i got myself a web site scraper and i m downloading right now... up to 3 gigs and coming !

thanks for the help!

ps: Files seems quite large, so this might not be the best idea, but i ll let it go and see how big it gets...

Last edited by Student1; 03-21-2009 at 03:47 PM.
Student1 is offline   Reply With Quote
Old 03-21-2009, 07:55 PM   #7
mjh215
Guru
mjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 989
Karma: 12653
Join Date: Apr 2008
Device: None of your business
There are roughly 1 million PD books on Google, if we average the PDF size at say 10 megs, you are talking 10TB of data, you might be biting off more then you can chew...

-MJ
mjh215 is offline   Reply With Quote
Old 03-21-2009, 08:15 PM   #8
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
I was thinking about that... 5 gigs and counting now! There are alot of german and spanish books i ll just delete. Well see when i start needing space... left one 36o gig hd just for this... once it gets to 100 gigs i ll have to do something

Wished the versions available would be the epub ones... as these pdfs will need some pdfread love to convert!

Last edited by Student1; 03-21-2009 at 08:19 PM.
Student1 is offline   Reply With Quote
Old 03-22-2009, 05:06 AM   #9
mjh215
Guru
mjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 989
Karma: 12653
Join Date: Apr 2008
Device: None of your business
Hmm, someone is welcome to check my math but if it took you 5 hours to go from 3 gigs to 5 gigs, that is 1GB per 2.5 hours, with 10TB roughly to go you should be done in about 5 to 6 months... But I like the idea, if I had the space and the bandwidth (I'm not -that- patient) I'm sick enough to do it too...

I'd rather have the PDF's then the epubs for archiving, you can always reOCR the PDF's or use them as visual reference for correcting an epub whereas with the epub you are stuck with the original OCR...

-MJ
mjh215 is offline   Reply With Quote
Old 03-22-2009, 05:10 AM   #10
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
Quote:
Originally Posted by mjh215 View Post
Hmm, someone is welcome to check my math but if it took you 5 hours to go from 3 gigs to 5 gigs, that is 1GB per 2.5 hours, with 10TB roughly to go you should be done in about 5 to 6 months... But I like the idea, if I had the space and the bandwidth (I'm not -that- patient) I'm sick enough to do it too...

I'd rather have the PDF's then the epubs for archiving, you can always reOCR the PDF's or use them as visual reference for correcting an epub whereas with the epub you are stuck with the original OCR...

-MJ
Yeah its a bit slow, about 7 gigs now... Just letting it slowly download! But its really nice they didn t think of protecting the web site from a scraper... guess they didn t think someone was crazy enough !

I wonder what they used to automate the ocr process of all those books. Can't believe they manually did them all... wouldn t make sense!
Student1 is offline   Reply With Quote
Old 03-22-2009, 05:31 AM   #11
mjh215
Guru
mjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 989
Karma: 12653
Join Date: Apr 2008
Device: None of your business
I imagine since they aren't listing a software partner anywhere that they are using their own, which is Tesseract.

-MJ
mjh215 is offline   Reply With Quote
Old 03-22-2009, 09:58 AM   #12
thibaulthalpern
Evangelist
thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.thibaulthalpern ought to be getting tired of karma fortunes by now.
 
Posts: 478
Karma: 451808
Join Date: Feb 2009
Location: California, USA
Device: my two eyes, KLiiK, Sony PRS-700
Wow....I don't believe anyone would collect digital books like this. Why are you doing it? It's so curious, from my point of view It's not the same as collecting physical books, is it? So many books on Google Books that I've searched under the keywords "ethnography", "anthropology" and the like are just crappy (excuse me) society transactions and old stuff that are out-dated now in the field.
thibaulthalpern is offline   Reply With Quote
Old 03-22-2009, 10:07 AM   #13
mjh215
Guru
mjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermjh215 can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 989
Karma: 12653
Join Date: Apr 2008
Device: None of your business
I have perhaps 50-100 texts from Google I've grabbed for reference. Many of which are texts that haven't been reprinted and the originals are quite rare. But I do like the idea of having anything available... And I've seen things disappear from access way too many times to go with the logic that it'll be there when you need it... Everyone's tastes and needs are different....

-MJ
mjh215 is offline   Reply With Quote
Old 03-22-2009, 09:20 PM   #14
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
Personally i want to create a very very good digital library. Like mjh215 said, its there now but who knows if it will always be there! These digital copies are almost better than the real thing, these never get old while books age and need to be replaced eventually.

Old books are like an inprint of time, sure archeology, history views might have changed, but its almost a social study to see how society thought at one time. Hence why these book are of much value to me !

Last edited by Student1; 03-22-2009 at 09:29 PM.
Student1 is offline   Reply With Quote
Old 03-22-2009, 09:23 PM   #15
Student1
Groupie
Student1 doesn't litterStudent1 doesn't litter
 
Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
Quote:
Originally Posted by mjh215 View Post
I imagine since they aren't listing a software partner anywhere that they are using their own, which is Tesseract.

-MJ
Thanks for the information!!
Student1 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Google Books Offers ePub Downloads Of Free Books AprilHare News 19 05-17-2011 10:02 PM
Sony, Google and Barnes & Noble To Partner For Sales of Google Books [April Fools] NatCh News 73 04-07-2009 08:48 AM
Five free books for download MsAstoria Amazon Kindle 16 03-05-2009 12:41 PM
Google Download: No iTunes for Books - BusinessWeek SoCal Bob News 5 01-24-2007 07:31 PM
Google to sell e-books that you cannot download Alexander Turcic News 4 03-14-2006 03:46 AM


All times are GMT -4. The time now is 05:44 PM.


MobileRead.com is a privately owned, operated and funded community.