track down the unknown exception in flightcrew

Sergey Glazyrin · 02-27-2018, 08:21 PM

Hello guys!
We generate a lot of epubs built from public sources (ie, internet sites, books, etc) and around 1 of 3 flightcrew validation fails with std::exception message. It's a kind of generic error. Nothing specific, I tried to track it down, localizing what kind of page is a cause of this problem. But unfortunately, no lack.
Could you please suggest how do I debug flightcrew compiled sources and find it out ?
I don't want to use strace as it is very hard to read and understand. Wouldn't be better to add a logging system into sources, compile again flightcrew and try to debug the problem.... ?
Thanks in advance

DiapDealer · 02-27-2018, 08:44 PM

Quote:

Originally Posted by Sergey Glazyrin

Wouldn't be better to add a logging system into sources, compile again flightcrew and try to debug the problem.... ?

Better for who?

But seriously, I've never encountered an exception with Flightcrew. So 1 out of 3 sounds extremely high. Are you using the Flightcrew plugin for Sigil, or are you compiling your own command-line version?

Sergey Glazyrin · 02-27-2018, 08:48 PM

Quote:

Originally Posted by DiapDealer

Better for who?

But seriously, I've never encountered an exception with Flightcrew. So 1 out of 3 sounds extremely high. Are you using the Flightcrew plugin for Sigil, or are you compiling your own command-line version?

By "better" I meant "faster". And yes it's for me

I use the version from github: https://github.com/Sigil-Ebook/flightcrew
and just compile it according to the documentation, without gui.

DiapDealer · 02-27-2018, 08:57 PM

If you can share a sample that generates the exception, it would probably be easiest to post a link to that sample in Flightcrew's github issue tracking system.

KevinH · 02-27-2018, 10:02 PM

Also, if you run the official flightcrew Sigil plugin we released, with our official binaries as a plugin, are you seeing the same high exception error rate?

Sergey Glazyrin · 02-28-2018, 02:15 AM

epub
strace
Would you suggest me to create a new issue and discuss with core flightcrew developers ?
In strace, to be honest, I see nothing special... except No such file or directory,

Maybe only this one is suspicious for me (I am not adding any of these files but flightcrew tries to get some information from these files):

Quote:

stat("/tmp/flightcrew/a462-6a34-5390-c6f2-a470-304c-bd25-fb22/META-INF/signatures.xml", 0x7ffe55f44870) = -1 ENOENT (No such file or directory)
stat("/tmp/flightcrew/a462-6a34-5390-c6f2-a470-304c-bd25-fb22/META-INF/encryption.xml", 0x7ffe55f44870) = -1 ENOENT (No such file or directory)

But definitely I've checked another epub which passes flightcrew validation, it tries to read this file as well

I use latest release 0.9.1 (from github)

Sergey Glazyrin · 02-28-2018, 02:16 AM

Quote:

Originally Posted by KevinH

Also, if you run the official flightcrew Sigil plugin we released, with our official binaries as a plugin, are you seeing the same high exception error rate?

I'll give it a try today

Doitsu · 02-28-2018, 03:22 AM

Quote:

Originally Posted by Sergey Glazyrin

I use latest release 0.9.1 (from github)

The latest version is 0.9.2. I've tested your book with the 64bit version of Sigil 0.9.9 and the pre-compiled 64bit Windows and Linux binaries and wasn't able to reproduce your problem.

You might want to give the pre-compiled binaries a try.

BTW, what's your OS and what Sigil version did you use for your tests?

Sergey Glazyrin · 02-28-2018, 06:03 AM

Quote:

Originally Posted by Doitsu

The latest version is 0.9.2. I've tested your book with the 64bit version of Sigil 0.9.9 and the pre-compiled 64bit Windows and Linux binaries and wasn't able to reproduce your problem.

You might want to give the pre-compiled binaries a try.

BTW, what's your OS and what Sigil version did you use for your tests?

Hello, thank you for your reply
Our production code hosted on Debian 9 Stretch OS. I don't use sigil at all because we need only a validation for our epubs before we upload it to Amazon to sell it.
Maybe I missed latest release... I'll check it out again.

DiapDealer · 02-28-2018, 06:15 AM

I don't believe any of the 0.9.2 changes affected the cli version. Those changes were all for the plugin binary if I recall correctly. I doubt if the version number for the cli version was even advanced at that time.

From the strace alone, it looks to me that the error is occurring while cleaning up the temp files flightcrew created.

I'll see if I can get it to crash on my machine, where I have a debug version of flightcrew-cli built. Perhaps a gdb backtrace will be more revealing.

Doitsu · 02-28-2018, 06:30 AM

Quote:

Originally Posted by Sergey Glazyrin

I don't use sigil at all because we need only a validation for our epubs before we upload it to Amazon to sell it.

FlightCrew catches many issues, however, it's somewhat outdated by now. You might want to consider using epubcheck instead. (epubcheck is a Java command line tool.)

JSWolf · 02-28-2018, 07:17 AM

There is a Sigil plugin for epubcheck. The latest version of epubcheck is 4.0.2.

DiapDealer · 02-28-2018, 07:42 AM

I can verify the exception on Linux with that epub, but very little else. gdb reports no stack when trying to backtrace.

KevinH · 02-28-2018, 07:45 AM

I will try it on OSX to see if lldb provides any more info.

DiapDealer · 02-28-2018, 09:17 AM

Correct me if I'm wrong, but isn't the main difference between flightcrew-cli and flightcrew-plugin the fact that the latter takes a directory as an argument, and the former takes an epub?

If that's the case (and I can verify that a newly built filghtcrew-plugin can handle the contents of this problematic epub) wouldn't that suggest the problem lies somewhere in flightcrew-cli's unzip-to-tmp routines?

02-27-2018, 08:21 PM	#1
Sergey Glazyrin Junior Member Posts: 8 Karma: 10 Join Date: Feb 2018 Device: Nexus 9	track down the unknown exception in flightcrew Hello guys! We generate a lot of epubs built from public sources (ie, internet sites, books, etc) and around 1 of 3 flightcrew validation fails with std::exception message. It's a kind of generic error. Nothing specific, I tried to track it down, localizing what kind of page is a cause of this problem. But unfortunately, no lack. Could you please suggest how do I debug flightcrew compiled sources and find it out ? I don't want to use strace as it is very hard to read and understand. Wouldn't be better to add a logging system into sources, compile again flightcrew and try to debug the problem.... ? Thanks in advance

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Standalone FlightCrew	capidamonte	Sigil	8	04-25-2012 05:20 PM
Is this a FlightCrew bug?	JSWolf	Sigil	5	10-04-2011 04:01 AM
Bug in FlightCrew	JSWolf	Sigil	11	07-30-2011 04:12 AM
Standalone flightcrew?	bfollowell	Sigil	4	06-30-2011 11:21 AM
FlightCrew and Norton IS	bobcdy	ePub	5	11-16-2010 05:28 PM

02-27-2018, 08:57 PM	#4
DiapDealer Grand Sorcerer Posts: 28,364 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	If you can share a sample that generates the exception, it would probably be easiest to post a link to that sample in Flightcrew's github issue tracking system.

02-27-2018, 10:02 PM	#5
KevinH Sigil Developer Posts: 8,487 Karma: 5703586 Join Date: Nov 2009 Device: many	Also, if you run the official flightcrew Sigil plugin we released, with our official binaries as a plugin, are you seeing the same high exception error rate?

02-28-2018, 06:15 AM	#10
DiapDealer Grand Sorcerer Posts: 28,364 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	I don't believe any of the 0.9.2 changes affected the cli version. Those changes were all for the plugin binary if I recall correctly. I doubt if the version number for the cli version was even advanced at that time. From the strace alone, it looks to me that the error is occurring while cleaning up the temp files flightcrew created. I'll see if I can get it to crash on my machine, where I have a debug version of flightcrew-cli built. Perhaps a gdb backtrace will be more revealing.

02-28-2018, 07:17 AM	#12
JSWolf Resident Curmudgeon Posts: 79,188 Karma: 144286760 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	There is a Sigil plugin for epubcheck. The latest version of epubcheck is 4.0.2.

02-28-2018, 07:42 AM	#13
DiapDealer Grand Sorcerer Posts: 28,364 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	I can verify the exception on Linux with that epub, but very little else. gdb reports no stack when trying to backtrace.

02-28-2018, 07:45 AM	#14
KevinH Sigil Developer Posts: 8,487 Karma: 5703586 Join Date: Nov 2009 Device: many	I will try it on OSX to see if lldb provides any more info.

02-28-2018, 09:17 AM	#15
DiapDealer Grand Sorcerer Posts: 28,364 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Correct me if I'm wrong, but isn't the main difference between flightcrew-cli and flightcrew-plugin the fact that the latter takes a directory as an argument, and the former takes an epub? If that's the case (and I can verify that a newly built filghtcrew-plugin can handle the contents of this problematic epub) wouldn't that suggest the problem lies somewhere in flightcrew-cli's unzip-to-tmp routines?

Advert

Advert