web2lrf

[Deputy-Dawg]

Kovid,
Just a tickler - I don't know how you monitor the fora.

[kovidgoyal]

Not too worry a bug fixed re-release of the beta will be heading out either later today or tomorrow.

[kovidgoyal]

New build are up with a bunch of bug fixes and a new NYTimes recipe that downloads only the current day's paper in under 3mins (on a fast connection/computer).

Note that using custom profiles from the GUI is still broken.

[Deputy-Dawg]

Kovid,
Down loaded the latest beta and the The New York Times does indeed download. But I am unable to open the file in the lbprs500 viewer. The progress bar goes to about 90% completion and then the program simply quits loading, or at least there is no further advancement of the progress bar thought it does still responds to input from the mouse and keyboard. The file does load on the PRS-505. Probably not a fatal error but one which might tend to produce a support headache.

BTW I get the same result whether the output file is created in the GUI or from the command line.

[kovidgoyal]

Hmm works for me. Did you download it with a correct username and password?

[Deputy-Dawg]

Yes, and the file does work correctly in the Sony Reader. It is only in the reader built into lbprs500 were it hangs. Weird!

Even stranger - I went back and loaded the last released version of lbprs500 (4.42) and its reader loaded the file just fine. So then to be sure that the problem was 'real' i tried it in the beta and sure enough it hung. I tried to visually capture the versions of the reader but they simply load to quickly for me to get the version numbers.

[kovidgoyal]

version 0.4.43 should hit the servers in a bit, with the updated code used throughout. There are probably still bugs, though I've tested most of the commonly used profiles.

[touser]

Call me stupid but i cant find the beta download directory on your website? Is there one, or an svn? Also, is there a changelog posted anywhere? Thanks for creating such great software, without libprs500 i would have returned my sony reader to the store, now i use it everyday.

[Deputy-Dawg]

Kovid,
I just down loaded 4.43 and a quick run shows that all of the problems I had noted have been resolved.

There is an issue, and it would not surprise me that there is no good fix, the 'headlines' in USA Today are huge. The body type in the articles are fine. In fact setting the basefont size wouldn't work because if you set it small enough to fix the headlines the body text would be much to small. Seems to me that it would be case for a regexp that would set the font size smaller if it is larger than X. But I haven't a clue as to how to write such a conditional.

And while I am about it, how difficult would be to look at the system clock and gather a group of feeds at a set time. To be able to do that would be a godsend on the days that I have hemodialysis. Imagine, when I get up at 4:30 the morning paper would be there in my laptop right along side my coffee from the automatic coffee pot.

I can dream can't I?

[kovidgoyal]

Use version 0.4.43, it's newer than the betas. And there's a link to the changelog next to the download link on the libprs500 website.

[kovidgoyal]

@DD

Actually using the new "recipes" framework, you don't even need regexps to fix USA Today, you can use the new extra_css property to override the font size for headings.

As for, a download scheduler, it's on my TODO list, but my TODO list is so long, that I really can't make any sort of commitment as to when it will get done.

In the meantime, you can always use cron on an OS X system with the commandline tools to schedule automatic downloads.

[kovidgoyal]

Here's the USAToday recipe:

from libprs500.web.feeds.news import BasicNewsRecipe
import re

class USAToday(BasicNewsRecipe):

    title = 'USA Today'
    timefmt  = ' [%d %b %Y]'
    max_articles_per_feed = 20
    no_stylesheets = True
    extra_css = '''
    .inside-head { font: x-large bold } 
    .inside-head2 { font: x-large bold }
    .inside-head3 { font: x-large bold }
    .byLine { font: large }
    '''
    html2lrf_options = ['--ignore-tables']

    preprocess_regexps = [
        (re.compile(r'<BODY.*?<!--Article Goes Here-->', re.IGNORECASE | re.DOTALL), lambda match : '<BODY>'),
        (re.compile(r'<!--Article End-->.*?</BODY>', re.IGNORECASE | re.DOTALL), lambda match : '</BODY>'),
        ]
    
    feeds =  [
                ('Top Headlines', 'http://rssfeeds.usatoday.com/usatoday-NewsTopStories'),
                ('Sport Headlines', 'http://rssfeeds.usatoday.com/UsatodaycomSports-TopStories'),
                ('Tech Headlines', 'http://rssfeeds.usatoday.com/usatoday-TechTopStories'),
                ('Travel Headlines', 'http://rssfeeds.usatoday.com/UsatodaycomTravel-TopStories'),
                ('Money Headlines', 'http://rssfeeds.usatoday.com/UsatodaycomMoney-TopStories'),
                ('Entertainment Headlines', 'http://rssfeeds.usatoday.com/usatoday-LifeTopStories'),
                ('Weather Headlines', 'http://rssfeeds.usatoday.com/usatoday-WeatherTopStories'),
                ('Most Popular', 'http://rssfeeds.usatoday.com/Usatoday-MostViewedArticles'),
                ]
    
    ## Getting the print version 
    
    def print_version(self, url):
        return 'http://www.printthis.clickability.com/pt/printThis?clickMap=printThis&fb=Y&url=' + url

[ddavtian]

Kovid, it's again me and the profile for WSJ.

With your help I got the profile working with web2lrf. When trying feeds2lrf, I get an error message about "time:

File "<string>", line 29, in WallStreetJournalPaper
NameError: name 'time' is not defined
Traceback (most recent call last):
File "convert_from.py", line 71, in <module>
File "convert_from.py", line 56, in main

This is the line 29 from profile:
issue_date = time.ctime()

Later "issue_date" is used here:
articles.append({
'title':title, 'url':url, 'description':'', 'date':issue_date
})


Can you tell me what to put here for it to work again?

[kovidgoyal]

Add

import time

at the top

[ddavtian]

"import time" is already there and it works with web2lrf.