Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Other formats > LRF

Notices

Reply
 
Thread Tools Search this Thread
Old 07-13-2007, 08:44 PM   #1
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
web2lrf

Building on my work with web2disk, here's web2lrf (part of libprs500 v0.3.70).

It directly converts websites into LRF files. More than that it has support for profiles that allow it to preprocess websites to generate better looking LRF files. Right now it knows about the New York Times, The BBC, The Economist and Newsweek (see attached demos).

To use it with a profile:
Code:
web2lrf profilename
For e.g. for newsweek
Code:
web2lrf newsweek
For The New York Times
Code:
web2lrf --username myusername --password mypassword nytimes
To create your own profile and use it with web2lrf visit
https://libprs500.kovidgoyal.net/wiki/UserProfiles for instructions and examples.

To use it with an arbitrary website (it wont do any preprocessing)
Code:
web2lrf --url http://mywebsite.com default
Enjoy!
Attached Files
File Type: lrf Newsweek [14 Jul 2007].lrf (2.42 MB, 1118 views)
File Type: lrf The New York Times [Thu, 16 Aug, 2007].lrf (1.21 MB, 1142 views)
File Type: lrf The BBC [Sun, 19 Aug, 2007].lrf (463.4 KB, 955 views)
File Type: lrf The Economist [27 Nov 2007].lrf (3.45 MB, 891 views)

Last edited by kovidgoyal; 11-28-2007 at 02:23 AM.
kovidgoyal is offline   Reply With Quote
Old 07-14-2007, 04:52 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Released v0.3.62 with a newsweek profile. See the attached demo in the first post. Since I actually read Newsweek, I've taken a little more pain over this profile, it has a nice hierarchical TOC.

Ironic since for some reason I haven't been getting my newsweeks for the past month ;-)
kovidgoyal is offline   Reply With Quote
 
Advertisement
Old 07-14-2007, 06:17 PM   #3
RWood
Technogeezer
RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.
 
RWood's Avatar
 
Posts: 7,233
Karma: 1601464
Join Date: Nov 2006
Location: Virginia, USA
Device: Sony PRS-500
I quickly uninstalled the prior version and installed the new one. Tried it.

"TypeError: option_parser takes() takes no arguments (1 given)"

Even the nytimes demo gave the same result.
RWood is offline   Reply With Quote
Old 07-14-2007, 06:26 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Oops typo...released 0.3.73 with the fix. It'll take 20 mins to reach the servers.
kovidgoyal is offline   Reply With Quote
Old 07-15-2007, 02:03 AM   #5
ddavtian
Addict
ddavtian has a complete set of Star Wars action figures.ddavtian has a complete set of Star Wars action figures.ddavtian has a complete set of Star Wars action figures.ddavtian has a complete set of Star Wars action figures.
 
Posts: 262
Karma: 332
Join Date: Nov 2003
Location: San Francisco, USA
Device: Sony 505 & 900, Kindle DX, Samsung Galaxy Tab, EVO
Kovidgoyal, thanks a lot for all your work.
NYTimes and BBC work fine for me, Newsweek gives an error message.
I'm looking forward for more profiles :-)

C:\Temp>web2lrf newsweek
Fetching feeds... done
Downloading .WARNING: Could not fetch link file://c:\docume~1\davidd~1\locals~1\
temp\libprs500oezp07\index.html

file://c:\docume~1\davidd~1\locals~1\temp\libprs500oezp07 \index.html saved to
Traceback (most recent call last):
File "convert_from.py", line 124, in <module>
File "convert_from.py", line 116, in main
File "convert_from.py", line 74, in create_lrf
File "libprs500\ebooks\lrf\html\convert_from.pyo", line 1233, in process_file
File "libprs500\ebooks\lrf\html\convert_from.pyo", line 1431, in get_path
File "libprs500\__init__.pyo", line 74, in extract
Exception: Unknown archive type
ddavtian is offline   Reply With Quote
Old 07-15-2007, 03:48 AM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
windows strikes again! fixed in 0.3.74.

As for new profiles, nothing planned as my needs are met. But feel free to contribute :-)
kovidgoyal is offline   Reply With Quote
Old 07-15-2007, 09:33 AM   #7
Platapie
Junior Member
Platapie began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jul 2007
Device: Sony PRS-500
You my friend, are a god. I just ordered my PRS-500 (not here yet but I can't wait), and was preemptively searching for ways to do this-- realize I'm jumping the gun here, but thanks so much.

Also great to see it offered for Linux
Platapie is offline   Reply With Quote
Old 07-15-2007, 09:59 AM   #8
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 37,893
Karma: 18755150
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Sony Reader PRS-650, iPad, nook STR
Quote:
Originally Posted by kovidgoyal View Post
windows strikes again! fixed in 0.3.74.

As for new profiles, nothing planned as my needs are met. But feel free to contribute :-)
Version 0.3.74 is not up on the website. Only 0.3.73.
JSWolf is online now   Reply With Quote
Old 07-15-2007, 12:50 PM   #9
RWood
Technogeezer
RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.RWood ought to be getting tired of karma fortunes by now.
 
RWood's Avatar
 
Posts: 7,233
Karma: 1601464
Join Date: Nov 2006
Location: Virginia, USA
Device: Sony PRS-500
Still coming up 0.3.73 for me.
RWood is offline   Reply With Quote
Old 07-15-2007, 02:36 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Sorry uploading now.
kovidgoyal is offline   Reply With Quote
Old 07-15-2007, 02:39 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by Platapie View Post
You my friend, are a god. I just ordered my PRS-500 (not here yet but I can't wait), and was preemptively searching for ways to do this-- realize I'm jumping the gun here, but thanks so much.

Also great to see it offered for Linux
It was developed for linux, windows and OSX support came later. Indeed it would not have been possible without all the other great free software that's been developed for linux.
kovidgoyal is offline   Reply With Quote
Old 07-16-2007, 05:17 PM   #12
ddavtian
Addict
ddavtian has a complete set of Star Wars action figures.ddavtian has a complete set of Star Wars action figures.ddavtian has a complete set of Star Wars action figures.ddavtian has a complete set of Star Wars action figures.
 
Posts: 262
Karma: 332
Join Date: Nov 2003
Location: San Francisco, USA
Device: Sony 505 & 900, Kindle DX, Samsung Galaxy Tab, EVO
Kovidgoyal, Newsweek now works for Windows, but the output is a rss type small file. Articles are not pulled.

I have few basic questions, sorry couldn't find in many pages of stickies.
How can I make the font smaller? Do I need to load more (or different) fonts to the reader or it's a setting somewhere for web2lrf?

Where can I find the profiles for NYTimes (or BBC or Newsweek)? I'd like to use it as a good template for other sites.

Thanks in advance,
David (not a power user)
ddavtian is offline   Reply With Quote
Old 07-16-2007, 08:33 PM   #13
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You can see the profiles here https://libprs500.kovidgoyal.net/bro...web?order=name

The newsweek error will be fixed in the next release.
kovidgoyal is offline   Reply With Quote
Old 07-20-2007, 03:06 PM   #14
Platapie
Junior Member
Platapie began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jul 2007
Device: Sony PRS-500
Quote:
It was developed for linux, windows and OSX support came later. Indeed it would not have been possible without all the other great free software that's been developed for linux.
Fair enough. Well, what can I say? I finally got my machine/the time to play with it and it's fantastic. The only issue I had was installing convertlit from source-- it needs, according to the README "LIBTOMMATH" to compile but the address listed in the README is incorrect and instead one needs to get LibTom from http://libtom.org/.

Nonetheless, once that was done I tried out the NYTime script and it worked like a charm. Thank you so much for the hard work.
Platapie is offline   Reply With Quote
Old 07-20-2007, 05:53 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,346
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You only need convertlit if you plan on converting lit files. The rest of libprs500 will work just fine without it.
kovidgoyal is offline   Reply With Quote
Reply

Tags
libprs500, web2lrf

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
web2lrf to capture blog archive? Deputy-Dawg Sony Reader Dev Corner 1 02-15-2008 12:41 AM
web2lrf: La Repubblica alexxxm Sony Reader 1 11-13-2007 01:27 PM


All times are GMT -4. The time now is 01:28 PM.


MobileRead.com is a privately owned, operated and funded community.