Bug in Kobo processing of epub files causing hang in "Processing content"

[BCooktheirish]

hi everyone,

i've got the same problem but i've no answer for you.

I was just wondering if you guys would like to creat a website to share our epubs!?

Because for exemple i would be interested in you stanford's articles since i've got courses in this university but i i've got the time to select articles etc...

Thank you guys.

B.C

[Elfuente]

Originally Posted by BensonBear

Yesterday I converted about 220 articles from the Standford Encyclopedia of Philosophy into epub and transferred them to my Kobo. I was not surprised to see that it appeared to get stuck somewhere "Processing Content". By the very tedious and laborious process described by others I was eventually able to transfer all but two of the files.

This really is unacceptable. Perhaps the Kobo team is unable to prevent their "Processing Content" stage from freezing. But this should provide better information to help people around their inadequate software. Instead of saying what percentage it thinks it has completed, the Kobo can tell us what file it is currently processing. We can then highly suspect that the file it last reports is the one it is stuck on. And it can provide a list of the files it processed that it had no problem with. It could probably even actually transfer those files instead of tranferring NO files if has trouble with one file. (If you get to "90%" for example, on 200 files, NO file at all will be transferred after all that work!)

Putting this issue aside, what was the problem with those two files? I looked into the original html trying to find something that those files had and none of the other 220 files had. And indeed there was something: the value of the "name" attribute in various <a> files has colons in them. For references, for example "Carnap:23a" was used as a reference to a paper Carnap published in 1923. This was carried over to Calibre's command line ebook-convert epub files. I don't know if this is allowed in html or epub's xhtml or not, except that every program I tried does accept it: all browsers, all epub readers I have on my pc.

The only thing that did not like it, and could not tell me why, and either hung or crashed in processing such a file was the Kobo.

I changed all these attribute values and the Kobo finally allowed me to add the files.

A couple of other comments: even if colons are not allowed (I don't feel too much like checking this out now, but I can't imagine why they are not), it is not because of that that Kobo failed. Otherwise it should just tell is there is a syntax error in the epub file. It should give much better behavior in this case, not a hang or crash.

Second: it took me a lot longer to do this because once I found the first file it could not handle (about fifty files in), I put it aside and kept going with the laborious trial and error process until I got all but one of the remaining files on. Only then did I look at the two files. Had I taken the first problematic file when I found it, and looked for an anomaly in it, I might have been able to avoid much of the rest of the long process. Hopefully others may take a smarter path in this regard.

However, Kobo must fix this. Not just the problem with colons, but also the messages it gives just before it crashes or hangs on an epub it cannot handle.

+1

I was yesterday struggling to find out which of my files cause KG to freeze during the content processing. What the developers of Kobo here delivered is really absolute crap, That´s ridiculous bad work.

[BensonBear]

Originally Posted by Elfuente

I was yesterday struggling to find out which of my files cause KG to freeze during the content processing. What the developers of Kobo here delivered is really absolute crap, That´s ridiculous bad work.

I believe someone has posted somewhere that if you look at the most recent files in the images directory you will be likely to be able to tell which file the kobo was on when it hung.

Sorry for not looking for the post... I no longer use the Kobo software except as a hook for my own pdf and epub renderers and database manager.

(btw, to clarify: large red font not in my original post but I assume added for emphasis in the quotation...)

[davidfor]

Originally Posted by BensonBear

I believe someone has posted somewhere that if you look at the most recent files in the images directory you will be likely to be able to tell which file the kobo was on when it hung.

Sorry for not looking for the post... I no longer use the Kobo software except as a hook for my own pdf and epub renderers and database manager.

(btw, to clarify: large red font not in my original post but I assume added for emphasis in the quotation...)

That might have been the case with the 1.9.x firmware. From 2.0.0, the cover images are generated the first time they are needed.

I did some experimenting recently about the processing order. It seems to be alphabetical. It is definitely alphabetical within a directory. When multiple directories are involved, it seems to be alphabetical as well. I haven't done a proper test on that, just checked the timestamps in the database after using SD card with about 500 books on it.

[TechniSol]

While I agree that ALL software should be able to process errors without abends (abnormal ends) and that more information than a screen that merely displays "processing" should be considered A REQUIREMENT, not an option, one does have to be realistic...

Reality #1: Ereaders are designed and sold as a conduit to further content from which the manufacturer expects to profit. We all know this, why are we so surprised? They aren't really sold to work with any file that just happens to have the right extension. I'm not saying that should let them off the hook, but it's not like the OP bought a Kobo book and couldn't get the reader to process it. Perhaps the OP's conversion process may play a role in the difficulty he is having? Nowhere did the OP detail the conversion or verification process, though there was a mention of calibre. Did he even run them through SIGIL or a similar product that can perform verification?

Reality #2: With number 1 in mind, perhaps the OP would get further requesting help from Kobo and making a suggestion regarding a more enlightening system than merely displaying "processing".

Reality #3: Rather than attempting to process in excess of two hundred files multiple times one might have divided them into manageable groups and attempted to process them so that the objectionable files could more easily be found by sub dividing until the errant files were discovered.

While I understand the OP's frustration, there were mitigating strategies as pointed out above...

[DNSB]

It sounds rather as if Calibre is using namespaces without defining them. You might want to look at the xmlns declaration at the start of the (x)html files within an epub as an example of a namespace definition. If you are going to use template:whatever for instance, you need to have a namespace definition for template.

Perhaps you might want to blame Calibre rather than Kobo for your issue.

[jusmee]

Originally Posted by DNSB

It sounds rather as if Calibre is using namespaces without defining them. You might want to look at the xmlns declaration at the start of the (x)html files within an epub as an example of a namespace definition. If you are going to use template:whatever for instance, you need to have a namespace definition for template.

Perhaps you might want to blame Calibre rather than Kobo for your issue.

Calibre might be causing errors, but Kobo still should crash ungracefully no matter what it finds. It;s OK for it not accept the malformed file, but to crash and burn with no clue is BAD programming. Especially as I have been reading complaints about it for a very long time, and many many versions of the software - and they still have never fixed it.