Viewer has problems when reading fetched news epubs

[kovidgoyal]

This should fix it:
https://github.com/kovidgoyal/calibr...c473f813b92099

Apparently the code to process EPUB files is now so fast the filesystem cant keep up with it.

[juanferna]

Thank you Kovid.
And congratulations for solving in such short time, because I guess that dealing with problems involving randomness is not an easy job.

As for the second problem indicated by unkn0wn in the first post of this thread (the second image, which refers to a missing file), I should say that it happened to me quite often some years ago. But nowadays, having a better internet connection, it rarely happens to me. Then, for me it's not an important problem.
Anyway, in case you decide some day to tackle this second problem, I have attached information that can help you.
At first glance, this second problem also involves randomness but that's not the case (when you already know how to trigger the error).

[DNSB]

Check the sample epub by cycling though it 3 times it and all looked good.

[juanferna]

You may not be clicking on the correct article.
This epub file fails on Art-1 -> Next and Art-3 -> Previous (and Art-4 -> Next).

This should be problematic regardless of the operating system (Windows, Linux, Android) or viewer/reader.
However, the behavior depends on the viewer/reader. In the case of the Calibre viewer, you will see the missing file screen.

[DNSB]

Originally Posted by juanferna

You may not be clicking on the correct article.
This epub file fails on Art-1 -> Next and Art-3 -> Previous (and Art-4 -> Next).

This should be problematic regardless of the operating system (Windows, Linux, Android) or viewer/reader.
However, the behavior depends on the viewer/reader. In the case of the Calibre viewer, you will see the missing file screen.

Sorry but I was testing the ePub you attached in message #8 in this thread. It does not fail using Thorium under Windows but did have issues using calibre's viewer. This issue was corrected by Kovid Goyal in the patch mentioned in his message #16 and no longer fails under calibre's viewer. I disregarded any references to attempts to link to files not in the ePub.

I looked at your missing articles from the .zip file linked in message #17 in this thread and the issue there is very simply that there are links to non-existent files. I opened the epub in Sigil and yes, I got errors. Perhaps that your reference to "<a href="../article_2/index.html" doesn't work since there article_2/index.html is not be be found in the ePub since only article_0, article_1, article_3 and article_4 exist in the epub. The same for <a href="../../feed_1/index.html"> since feed_1/index.html does not exist. Complaining that links to non-existent locations does not work is not going gain you much sympathy.

While I had the file open in Sigil, I ran epubcheck on it and that showed 4 errors and I have attached the image of the error messages and the files as shown in Sigil's file browser.

In future, I would suggest checking your ePub using epubcheck in calibre's editor before complaining about issues.

[kovidgoyal]

https://github.com/kovidgoyal/calibr...e4040d6e0b7b5d

[juanferna]

Originally Posted by DNSB

Sorry but I was testing the ePub you attached in message #8 in this thread. It does not fail using Thorium under Windows but did have issues using calibre's viewer. This issue was corrected by Kovid Goyal in the patch mentioned in his message #16 and no longer fails under calibre's viewer. I disregarded any references to attempts to link to files not in the ePub.

I looked at your missing articles from the .zip file linked in message #17 in this thread and the issue there is very simply that there are links to non-existent files. I opened the epub in Sigil and yes, I got errors. Perhaps that your reference to "<a href="../article_2/index.html" doesn't work since there article_2/index.html is not be be found in the ePub since only article_0, article_1, article_3 and article_4 exist in the epub. The same for <a href="../../feed_1/index.html"> since feed_1/index.html does not exist. Complaining that links to non-existent locations does not work is not going gain you much sympathy.

While I had the file open in Sigil, I ran epubcheck on it and that showed 4 errors and I have attached the image of the error messages and the files as shown in Sigil's file browser.

In future, I would suggest checking your ePub using epubcheck in calibre's editor before complaining about issues.

Sorry DNSB. I think there has been a misunderstanding about our messages.
When I saw your message #18, I thought it was the reply to my message #17. Then, I post my message #19 as a reply to your message #18.
But now I see your message #18 was your reply to Kovid message #16.
I shouldn't have posted message #19. My apologies.


After this mess, I think the situation is as follows:

• unkn0wn started this thread pointing out two different issues (the two images of message #1).
• As for the first issue (first image):
• • I posted message #8 attaching some information.
  • Kovid modified Calibre (message #16)
  • DNSB tested the modification (message #18)
• As for the second issue (second image):
• • I posted message #17 attaching some information.
  • DNSB post message #20 indicating the epub file Calibre has generated is badly formatted (links to non-existent files). I agree with this as I indicated in my attachment (post #17)
  • As I said, this second issue wasn't important for me and didn't expected a reply (at least, in the short term). In any case, what to do now?. I think there are several options:
    • 1. Do nothing: Calibre's current behaviour is valid (Calibre write warnings to its log indicating problems abut fetching. Therefore, users should decide what to do with the resulting epub file).
    • 2. Modify Calibre so that it doesn't generate the epub file: an error (not a warning) is generated when it's not possible to fetch a file.
    • 3. Modify Calibre so that it generates the epub file without inconsistencies.
    • 4. Modify Calibre (more specifically, Calibre-viewer) so that it reads the epub file despite the inconsistencies.
    • 5. Another solution.
  • In my opinion, the best option is number 3. But, anyway, it's Kovid Goyal's decision.

[DNSB]

The patch in message #21 now has calibre's viewer popping up an error message about the file not being found.

The problem with #4 is that you are asking for calibre's ebook-viewer to decide what to display when a link to a non-existent file is followed.

This type of situation is one the main reasons that I do not send epubs to my ereaders until I have checked and corrected any errors. Garbage In, Garbage Out. I tend to be a bit OCD with how I want epubs to look so I am happy to spend the time to correct issues.

OTOH, those links used in your ePub look like a web page from decades back and is not something that I've seen in quite a while on a web page never mind an ePub ebook. A ToC would be a better solution.

[juanferna]

Originally Posted by DNSB

OTOH, those links used in your ePub look like a web page from decades back and is not something that I've seen in quite a while on a web page never mind an ePub ebook. A ToC would be a better solution.

Calibre automatically generates those links when you create an epub file with a recipe.
They may look strange, but recipes are commonly used for news.
And I find those links helpful. In many articles, I just read the headline (and sometimes the beginning) and move on to the next one. You can do this simply by clicking the "Next" link.

This is how I encountered the first of the two issues mentioned in the current thread.
Since it was a bit annoying to restart calibre-viewer every now and then, I searched the forum to see if anyone had encountered the same problem and what the solution was. And I found this thread.
So my reasons for posting in the thread about the first issue were selfish.

But unkn0wn also mentioned the second issue. I was already familiar with it, and although it wasn't a problem for me anymore, I decided to include the information I had (mainly how the problem was triggered) in case it was useful to anyone. In this case, my reasons for posting about the second issue were altruistic, and I didn't expect anything.

Finally, thank you both, DNSB and, especially, Kovid, for your help.
I hope my posts have also been helpful.

[DNSB]

I found them helpful. Though I do not use recipes for fetching news, it is always interesting to see what others are doing.

[juanferna]

Originally Posted by kovidgoyal

https://github.com/kovidgoyal/calibr...e4040d6e0b7b5d

Tested with Calibre 8.4, it works fine.
When using the MissingArticle.epub file generated with Calibre 8.3, a popup appears and it doesn't gets stuck.
If the MissingArticle.epub file is generated again with Calibre 8.4, the new file shows no inconsistencies (there are no references to a missing internal file).

[juanferna]

Originally Posted by kovidgoyal

This should fix it:
https://github.com/kovidgoyal/calibr...c473f813b92099

Apparently the code to process EPUB files is now so fast the filesystem cant keep up with it.

Tested with Calibre 8.4, it still occasionally has issues.
However, 8.4's behavior is much better than 8.3's: in cases where it still doesn't work properly, the new popup appears and there's no need to restart the viewer, so I consider this a low-priority issue.
Additional information is attached.