MobileRead Forums
Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Readers > Sony Reader
Reload this Page web2disk

Notices

Reply
 
Thread Tools Search this Thread
Old 07-12-2007, 11:26 PM   #1
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,857
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
web2disk
Released libprs500 v0.3.69 with the program web2disk. It allows you to download websites to your harddisk in a format that's suitable to run through html2lrf to generate a nice LRF file for your SONY Reader.

Code:
web2disk http://www.google.com
It fetches links recursively, by default to one level, so you can potentially download an entire website.

Enjoy and report bugs. This is a few hours work, so there are bound to be many.
kovidgoyal is offline   Reply With Quote
Old 07-13-2007, 12:27 AM   #2
Nate the great
Sir Penguin of Edinburgh
Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.
 
Nate the great's Avatar
 
Posts: 12,375
Karma: 23555235
Join Date: Apr 2007
Location: DC Metro area
Device: Shake a stick plus 1
Hey! I'm gonna download the Internet!

web2disk *.*.*.*


But seriously, can you set a delay of one request per second? That is the cutoff for Wikipedia. Any faster and they will block the URL.
Nate the great is offline   Reply With Quote
Advert
Old 07-13-2007, 07:17 AM   #3
Pode
Member
Pode began at the beginning.
 
Pode's Avatar
 
Posts: 22
Karma: 10
Join Date: Mar 2007
Location: France
Device: Sony Reader
AdBlock (and AdBlock Plus, its successor), two Firefox extensions can use a list of regex to block ads from webpages.
I put here a link to a good list (http://adblock.free.fr/adblock.txt), which can be useful to people wanting to retrieve lare bunch of sites, without following ads and stats links.
Pode is offline   Reply With Quote
Old 07-13-2007, 01:03 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,857
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
When you say a good list you mean a list of websites that should not be blocked? Also why are there so many / in the regexps?
kovidgoyal is offline   Reply With Quote
Old 07-13-2007, 02:26 PM   #5
Pode
Member
Pode began at the beginning.
 
Pode's Avatar
 
Posts: 22
Karma: 10
Join Date: Mar 2007
Location: France
Device: Sony Reader
I mean a list of site that *should* be blocked, since all those regexps are matching sites known for their ad campaigns or stats placement code.
My post wasn't very clear...

By the way, the list wasn't done by me, but I use it everyday. And I have forgottent since a long time what's like to surf on pages with advertisements with this tool !
Pode is offline   Reply With Quote
Advert
Old 07-13-2007, 03:21 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,857
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by Nate the great View Post
Hey! I'm gonna download the Internet!

web2disk *.*.*.*


But seriously, can you set a delay of one request per second? That is the cutoff for Wikipedia. Any faster and they will block the URL.
There'll be a --delay option in the next release.
kovidgoyal is offline   Reply With Quote
Old 02-28-2009, 01:49 PM   #7
svakanda
Enthusiast
svakanda began at the beginning.
 
svakanda's Avatar
 
Posts: 31
Karma: 44
Join Date: Feb 2009
Device: none
This is awesome, thanks so much Kovid!
svakanda is offline   Reply With Quote
Reply

Tags
libprs500, web2disk

« Previous Thread | Next Thread »

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
web2disk - howto? urbane_gorilla Calibre 3 03-29-2010 04:48 AM
website crashes ebook-convert after web2disk eksor Calibre 1 03-11-2010 06:58 AM
chaining web2disk to html2lrf beowulf573 Calibre 2 11-19-2008 04:48 PM


All times are GMT -4. The time now is 12:00 AM.

Contact Us - FAQ - Guidelines - Privacy - Top

MobileRead.com is a privately owned, operated and funded community.