remove_tags_after with more values

[niederrhymer]

Hello,

I want to put feeds from more than a website into a single .epub. Basicly it works quiet good, but I run into problems with the details.

"feeds" obviously is an array:

PHP Code:

feeds = [
        (u'FAZ - Politik',u'http://www.faz.net/aktuell/politik/?rssview=1'),
        (u'RP - Duesseldorf',u'http://feeds.rp-online.de/rp-online/rss/duesseldorf-stadt?format=xml'),
        (u'International',u'http://www.nzz.ch/nachrichten/international?rss=true')    
] 


Also "remove_tags".

But "remove_tags_before" and "remove_tags_after" requiere an single value.

Is it possible to rewrite the code so that I can use an array as well? I may have to rewrite (overwrite) a function of "BasicNewsRecipe". But which one?

Please, can you help me?

I am new to phyton but I got some expirience in other programming languages, e.g. Java, VB, C#. So please feel free to talk a bit more technicaly.

regards
Tom

[roedi06]

But "remove_tags_before" and "remove_tags_after" requiere an single value.

Is it possible to rewrite the code so that I can use an array as well? I may have to rewrite (overwrite) a function of "BasicNewsRecipe". But which one?

Can you explain me what te point is of multiple "remove_tags_before" and "remove_tags_after". If you use more of those, you are basicly implementing "remove_tags". The functions "before" and "after" are a bonus to make life easier if you just want to keep a certain portion, without the hassle of removing every other tag. From withing those tags, you need to specify other remove_tags ofcourse.

[niederrhymer]

Yes, I can. Sorry for that, I thought it was obviously because I use more than one newssource.

I need this for the "Neue Züricher Zeitung" and for the "Rheinische Post" and certainly for some more.

All those newssources should be collected in _one_ single EPUB File.

Originally Posted by roedi06

Can you explain me what te point is of multiple "remove_tags_before" and "remove_tags_after".

"remove_tags" is quite good if you only want to remove some pics, or pic undertitle. It's a mess to substitute "remove_tags_after" and "remove_tags_before" with it.

If you use more of those, you are basicly implementing "remove_tags".

That's exactly what I want and I want it depending on my current source, e.g. "remove_tags_before = dict(id='headline')" for the Rheinische Post and "remove_tags_before = dict(name='p', attrs={'class':'dachzeile'})" for the NZZ. And I don't want to have two recipes and two epub files.

The functions "before" and "after" are a bonus to make life easier if you just want to keep a certain portion, without the hassle of removing every other tag. From withing those tags, you need to specify other remove_tags ofcourse.

Any ideas?

[roedi06]

I see what you mean now. I didn't read it well enough. For multiple feeds, it does make sense what you ask.

Can't you work with; IF this feed is selected than remove_before and after is THIS, and so on?

[kovidgoyal]

No, you cannot have per feed values in remove_tags_* . You will need to implement the cleanup yourself, in preprocess_html

[niederrhymer]

Thanks kovidgoyal,

I've seen "preprocess_html" yesterday by myself and started to cope with soup. If I got a solution, I'll post it here, but I think it'll take some time. Lerning Python + a new lib + less spare time.

I'll keep you up-to-date.

[sup]

Originally Posted by kovidgoyal

No, you cannot have per feed values in remove_tags_* . You will need to implement the cleanup yourself, in preprocess_html

If that is indeed the case, would you please update the documentation here ?

remove_tags_after shows this example:

remove_tags_after = [dict(id='content')]

which is actually wrong because if used like that (with the "[]" making it into a list), I got this:

TypeError: find() argument after ** must be a mapping, not list

Also, the documentation should note that even if the basic syntax is the same as with remove_tags, it must not be a list.

(BTW: I would love if one could use lists in these cases as well. I am writing a recipe for one magazine and it uses some special formatting for certain articles, so for those articles I have to somehow re-implement remove_tags_before myself, hopefully it wont be that hard:-)).

[kovidgoyal]

Lists containing dicts work perfectly well for remove_tag_after. That has nothing to do with them being per-feed.

remove_tag_after can be either a dict or a list containing dicts.

[sup]

Sorry, my bad. I meant remove_tag_before. True, the example for that is not a list, but it still links to the remove_tags syntax that says it takes a list of dicts. A sentence saying that remove_tag_before only accepts single dicts and not lists of them would be helpful (making it accepts lists would be even better:-)).

[kovidgoyal]

You can match multiple kinds of things with a single dict, but here you go:

https://github.com/kovidgoyal/calibr...dd72bb84cfd12e

[sup]

Thanks!