Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 05-25-2010, 04:42 PM   #1
maxbookworm
Addict
maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.
 
maxbookworm's Avatar
 
Posts: 367
Karma: 1122865
Join Date: Apr 2010
Device: Nook Glowlight, Galaxy Note II
Calibre + NYT download = inconsistent?

Hi all,

I have been using the NYT recipe in Calibre to download the full paper (I have a subscription). This is an awesome, amazing feature and I love it!

The thing is, about half the time the paper downloads, a portion of the articles are blank (usually in the Front Page section). Generally, if I re-download once or twice, this problem corrects itself and I get the full paper. I'm guessing this isn't a Calibre issue since sometimes it works and sometimes it doesn't. But has anyone else noticed this and tied it to anything specific? Is it the NYT server? Is it my internet connection? What could be causing this behavior and are there any tricks that will consistently overcome it?

Thanks!
maxbookworm is offline   Reply With Quote
Old 05-25-2010, 08:14 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,919
Karma: 5035037
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Try looking at the download log to see what error messages are generated when you notice blank articles. That should give us a clue on the root cause.

You can get the log by clicking the rotating spinner in the ottom right corner.
kovidgoyal is offline   Reply With Quote
Old 05-27-2010, 08:37 AM   #3
jsl21
Member
jsl21 began at the beginning.
 
Posts: 17
Karma: 10
Join Date: May 2010
Device: Kindle
I am having the same problem for the past few days with the NYT. If you look, the page actually says "This is an advertisement" so I am guessing what is happening is it goes to the website and an ad is being shown first before it shows the article. I am not sure how to edit the recipe to skip the ad which I would normally do on the website by clicking on skip the ad. Any ideas? By the way, sometimes you can find the actual articles from the front page in a different section, e.g., today, may 27th, the E Coli article I couldn't see when choosing it from the front page but it was in the Business section. Today I could find 3 out of the 5 that just said "this was an advertisement"
jsl21 is offline   Reply With Quote
Old 05-27-2010, 10:04 AM   #4
maxbookworm
Addict
maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.
 
maxbookworm's Avatar
 
Posts: 367
Karma: 1122865
Join Date: Apr 2010
Device: Nook Glowlight, Galaxy Note II
Thanks, Kovid. I will paste that log here next time it happens (of course, it was fine this morning). Jsl21 -- I don't get that "advertisement" message. It is just a blank page with the navigation links at top and article url at bottom. It's a good theory, though.
maxbookworm is offline   Reply With Quote
Old 05-27-2010, 10:12 AM   #5
jsl21
Member
jsl21 began at the beginning.
 
Posts: 17
Karma: 10
Join Date: May 2010
Device: Kindle
Kovid,

I still have the "This is an advertisement problem." I looked at the log and it seems to say that page downloaded without an error but I am not an expert so I cannot tell. I saved the log. Should I post it?
jsl21 is offline   Reply With Quote
Old 05-27-2010, 10:23 AM   #6
JRG
Member
JRG began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Mar 2010
Device: DR800SG
Quote:
Originally Posted by maxbookworm View Post
Thanks, Kovid. I will paste that log here next time it happens (of course, it was fine this morning). Jsl21 -- I don't get that "advertisement" message. It is just a blank page with the navigation links at top and article url at bottom. It's a good theory, though.
That's exactly what I get for most (maybe 75%) of the articles using the BBC recipe; I've pretty much abandoned BBC downloads for that reason. But the problem occurs rarely, if ever, using the NY Times recipe.
JRG is offline   Reply With Quote
Old 05-27-2010, 01:17 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,919
Karma: 5035037
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The advertisement problem would need a workaround in the recipe, the log wont help with that. IIRC the recipe used to handle that, but maybe I'm wrong.

I'll ping GRiker the author of the recipe to take a look.
kovidgoyal is offline   Reply With Quote
Old 05-27-2010, 03:14 PM   #8
kiklop74
Guru
kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.kiklop74 can program the VCR without an owner's manual.
 
kiklop74's Avatar
 
Posts: 780
Karma: 194642
Join Date: Dec 2007
Location: Argentina
Device: Kindle PaperWhite, Motorola Xoom
I noticed that BBC changed some things on their website hence the BBC Fast recipe was not working correctly.

Here is the updated version.
Attached Files
File Type: zip bbc_fast.zip (1.2 KB, 53 views)
kiklop74 is offline   Reply With Quote
Old 05-27-2010, 05:32 PM   #9
JRG
Member
JRG began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Mar 2010
Device: DR800SG
Quote:
Originally Posted by kiklop74 View Post
I noticed that BBC changed some things on their website hence the BBC Fast recipe was not working correctly.

Here is the updated version.
Thank you, very much!
JRG is offline   Reply With Quote
Old 05-28-2010, 09:20 AM   #10
maxbookworm
Addict
maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.
 
maxbookworm's Avatar
 
Posts: 367
Karma: 1122865
Join Date: Apr 2010
Device: Nook Glowlight, Galaxy Note II
OK jsl21, you're exactly right. I had never noticed that text before, but on the blank pages it says "This is an advertisement" and "Skip this ad" (see attached).

But usually if I re-download, the problem is fixed. Like maybe the second download gets you past the ad?
Attached Thumbnails
Click image for larger version

Name:	calibre_image.jpg
Views:	64
Size:	35.7 KB
ID:	52386  
maxbookworm is offline   Reply With Quote
Old 05-29-2010, 06:25 AM   #11
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,497
Karma: 424627
Join Date: Mar 2009
Device: iPad
I've submitted a patch to Kovid for both versions of the NYTimes recipes (Headlines, full paper) which should properly bypass the ads.

G
GRiker is offline   Reply With Quote
Old 05-29-2010, 07:29 AM   #12
maxbookworm
Addict
maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.maxbookworm ought to be getting tired of karma fortunes by now.
 
maxbookworm's Avatar
 
Posts: 367
Karma: 1122865
Join Date: Apr 2010
Device: Nook Glowlight, Galaxy Note II
Wow, you guys are amazing!

Thank you so so much for these programs. They are awesome!
maxbookworm is offline   Reply With Quote
Old 05-29-2010, 07:39 AM   #13
jsl21
Member
jsl21 began at the beginning.
 
Posts: 17
Karma: 10
Join Date: May 2010
Device: Kindle
Thanks GRiker!

maxbookworm - I also have found that rerunning it the same day will fetch a new copy without the "advertisement" problem. I guess the site remembers that you've already seen ads recently and doesn't force you to see them again
jsl21 is offline   Reply With Quote
Old 05-31-2010, 08:11 AM   #14
jsl21
Member
jsl21 began at the beginning.
 
Posts: 17
Karma: 10
Join Date: May 2010
Device: Kindle
GRiker,

Thanks for your help. I assume that your fixes to the NYT are in 0.6.55. Now it skips the first 4-5 articles on the front page (which I am guessing are the ones that come up with the "advertisement"). and they aren't downloaded at all. Was this what is supposed to happen?
jsl21 is offline   Reply With Quote
Old 05-31-2010, 08:21 AM   #15
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,497
Karma: 424627
Join Date: Mar 2009
Device: iPad
@jsl21: I just downloaded today's paper, and I got every article listed in the index. Are you comparing to the same index at http://www.nytimes.com/pages/todayspaper/index.html?

I believe that the ads are being skipped correctly, so I'm guessing that the problem you're experiencing is something else. It could be that some articles are being updated at the time you're running the recipe, or something else causing a connection failure during the download of that article.

Take a look at the download log to see if there's any further information there.

I'll keep poking around. It's possible that I'm just not getting any ads at this time.

G

Last edited by GRiker; 05-31-2010 at 08:53 AM.
GRiker is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Inconsistent Display of Cover Motto Kobo Reader 2 03-22-2011 11:36 PM
Inconsistent Metadata windybt Calibre 8 10-10-2010 12:31 PM
Calibre NYT News Fetch Problem e-literacy Calibre 1 04-01-2010 06:40 PM
Calibre, NYT and Opinion pages astromusic Calibre 0 03-02-2009 04:41 PM
Cannot Download Calibre 0.4.89 JSWolf Calibre 9 10-01-2008 06:06 PM


All times are GMT -4. The time now is 02:07 PM.


MobileRead.com is a privately owned, operated and funded community.