03-31-2014, 04:56 PM | #1 |
Guru
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
|
News articles repeat
Has anyone noticed that frequently articles will repeat in one download? I see this in the New York Times, the Boston Globe, Associated Press and others. It seems random, and to occur for news sources that have more (vs. fewer) articles per day.
|
03-31-2014, 06:48 PM | #2 |
Enthusiast
Posts: 32
Karma: 10
Join Date: Apr 2011
Device: Kindle wifi; Dell 2in1
|
Is this happening with the built-in recipes or custom recipes?
one thing to make sure that is in the recipe is the following. I know it works well with the few custom recipes I've been using. ignore_duplicate_articles = {'title', 'url'} remove_empty_feeds = True #Removes an empty feed in the download (especially if all duplicates). |
04-01-2014, 10:27 AM | #3 | |
Guru
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
|
Quote:
|
|
05-06-2014, 05:06 PM | #4 |
Guru
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
|
|
05-06-2014, 09:22 PM | #5 |
creator of calibre
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
SOme people like having repeated artricles in different sections, since they only browse a few sections. It is up to the authors of the recipes in question if they want to remove duplicates of not. If you disagree witht he recipe authros choice you are free to customize the builtin recipe for yourself.
|
05-07-2014, 09:19 AM | #6 | |
Guru
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
|
Quote:
Is there an easy set of command lines I can insert into the recipes to eliminate the dupes? Thanks. |
|
05-07-2014, 09:23 AM | #7 |
creator of calibre
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Camper65 already posted them for you.
|
05-07-2014, 09:42 AM | #8 |
Guru
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
|
|
06-04-2014, 03:40 PM | #9 | |
Guru
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
|
Quote:
|
|
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
TOC - Categories and articles like Calibre News | alexvf | Conversion | 4 | 05-08-2011 04:41 PM |
Feed content instead of articles in downloaded news | bubak | Recipes | 2 | 04-05-2011 06:38 PM |
The Age Feed - repeat articles | Quasii | Recipes | 2 | 03-09-2011 06:38 PM |
News Articles List in 0.7.20 | Ola | Calibre | 3 | 09-27-2010 07:38 AM |