Sigil-0.9.991 (1.0 Pre-release)

DiapDealer · 11-09-2019, 10:08 PM

Announcing Sigil-0.9.991 (pre Sigil-1.0) Alpha Release

----------------------------------------------------------------------------------

Before Sigil 1.0 can be released, we felt it was very important to remove the restriction that all epubs must be in Sigil standard form which included one OEBPS folder, specific names for the OPF, and NCX, only one folder for each type of resource (Text, Images, etc.).

Therefore, over the last two months, we have had to tear out much of the internals of Sigil to teach it to not standardize epubs on initial load, not to rename the OPF, NCX, not to move files to our specific folders, etc. Then we had to teach Sigil how to properly work with files in any structure or locations, how to build the correct links between them. This included extending the Sigil plugin interface appropriately.

To accomplish this over 4000 lines of code were added and over 2000 were deleted, touching almost every file in Sigil. All of this work has been very disruptive with the potential for lots of breakage and new bugs being introduced.

Special thanks must be given to both BeckyEbook and Doitsu for regularly building and testing the changes as they were being made and reporting back any breakage or issues they found and providing feedback on ideas. Without their hard volunteer effort, Sigil would be in a much much worse condition.

The work on Sigil-1.0 has now stabilized, and it is now ready for additional volunteers and testers to try and break it so that any final nits or issues can be tracked down and fixed before the upcoming Sigil 1.0 release.

The main changes include the following:

Upon initial load, the epub structure that exists will be loaded and kept in its "as-is" state, moving and renaming nothing
The BookBrowser will still show the Text, Images, Styles, Fonts, Audio, Video, and Misc "folders" but these are now "virtual folders" that simply organize the existing resources by type but do not restrict their actual location within the epub.
A new Preferences setting has been added to the Appearance Preferences that will determine if the book folder relative path will be shown in the BookBrowser or its shorter unique name (filename) will be used instead. In either case, the BookBrowser tooltips will always show the file's true book folder relative path. Duplicate file names (in different subfolders) will always show the longer path to be able to differentiate between them regardless of the new Appearance Preference.
A Tools menu item has been added to standardize any epub already loaded to Sigil's old structure
A Tools menu item has been added to allow a user to design a custom "empty" epub layout
A "Move" context menu item has been added to BookBrowser to enable moving of resources to where you want them to be.
A "Rename" context menu item has been added to BookBrowser to enable renaming of the NCX and OPF resources.
The python plugin interface has been extended to support all of this as documented in the new Sigil_Plugin_Framework_rev12.epub and online at MobileRead in our Plugin Development thread.

Additional bug fixes include:

handle javascript location.href style in Preview
use CaseInsensitive sort in Plugins menu (Thank you BeckDTP)
use Locale aware sort routines for AddMetadata, and MetaData Names (Thankyou Doitsu)
recognize x-font-otf style media types
recognize vnd.adobe.page-template+xml media type
recognize webp images media type
improve updating of smil xml files when moved
preserve whitespace inside the code element when prettifying
try again to prevent timer events during MainWindow close that causes random crashes on exit on macOS
fix svg cover images to use 100vh and 100vx in Preview

Please note, the design goal of this project was to make minimal changes for users and ebook developers that still want to use Sigil as they did. The new Preference setting defaults to showing only the filename as before, and the default empty epub for creating new epubs is the same as before. The interface and functionality should stay almost exactly the same. Loading any epub previously saved by Sigil will still work exactly as before. Only editing new epubs that are in non-standard form will necessitate a use of the new Tool menu (see #4 above) to standardize it.

Call For Feedback, Testing, Plugin Developers, etc...

----------------------------------------------------------------------------------

So we really need your help to test Sigil to make sure we have not broken anything and to provide feedback on the new features.

This release will also serve to allow Plugin Developers to test their plugins and make any changes needed to make sure their plugins will still work as expected.

So please take a minute to download Sigil-0.9.991 and test it as you would use earlier versions of Sigil, reporting back on MobileRead or via our GitHub site any bugs you find with instructions on how to recreate them. This is especially true of some of Sigil's more advanced features such as HTML TOC, Index generation, Merging and Splitting of Files, Reports, movement of resources, and any other feature of Sigil you would normally use on a regular basis. Though we're not going to ignore any reported issues, we're asking that you try to limit your reports to broken functionality or incorrect behavior. Cosmetic issues and minor inconveniences (with easy workarounds) can be dealt with later.

Note: Sigil-0.9.991 should not be considered a "production-ready" release. It is being made available to make sure our upcoming Sigil 1.0 release is as bug free and stable as possible.

Sigil-0.9.991 (1.0 pre-release) page and downloads.

un_pogaz · 11-10-2019, 02:32 AM

Yeah, well done on the job.

Now bug report: First ePub, first bug :
I have applied a manual restructuring "OEBPS/Text/Scheer,K.-H.&Darlton,Clark-[Perry Rhodan-001]Operation Astree (1961)_split_006.html" is named in the OPF "../Scheer%2CK.-H.%26amp%3BDarlton%2CClark-%5BPerry%20Rhodan-001%5DOperation%20Astree%281961%29_split_006.html" (old index before restructuring)
Consequently, it is deleted.
I suspect a special character because the files in [A-z] only are correctly restructured.

Also, I think that an option to automatically apply a structure (either Sigil's or the custom one) at the opening of any ePub, would be very useful (no ask user, old working mode).

Another good thing would be to be able to save several structures and be able to apply them easily (a "Structures" sub-menu).

PS: I don't know if you planned to add it to this version, but fortunately Sigil warns us when he encounters a none (or poorly) indexed files.

EDIT: the files with special character are not even renamed, under any circumstances!
EDIT2 : It's even more vicious, if I save the ePub even without renaming anything, the second opening will fail.

BetterRed · 11-10-2019, 04:29 AM

Any known reason why I shouldn't try it in the Windows 10 Pro Sandbox ?

BR

Doitsu · 11-10-2019, 04:32 AM

Quote:

Originally Posted by un_pogaz

Now bug report: First ePub, first bug :
I have applied a manual restructuring "OEBPS/Text/Scheer,K.-H.&Darlton,Clark-[Perry Rhodan-001]Operation Astree (1961)_split_006.html" is named in the OPF "../Scheer%2CK.-H.%26amp%3BDarlton%2CClark-%5BPerry%20Rhodan-001%5DOperation%20Astree%281961%29_split_006.html" (old index before restructuring)

I created a non-standard epub with the exact same file name and wasn't able to reproduce your bug.

Did you run EPUBCheck before your test?

Please post the following marked as spoilers:

1. The contents of the original .opf file.
2. The contents of the original .ncx file.
3. The contents of the new .opf file.
4. The contents of the new .ncx file.

DiapDealer · 11-10-2019, 04:34 AM

My god.

The ridiculousness of some filenames out there that need to be accommodated never ceases to amaze me. We'll look into it, of course, but... my god.

Also, I'm unclear what you mean by:

Quote:

PS: I don't know if you planned to add it to this version, but fortunately Sigil warns us when he encounters a none (or poorly) indexed files.

Doitsu · 11-10-2019, 04:59 AM

Quote:

Originally Posted by DiapDealer

Also, I'm unclear what you mean by [...]

AFAIK, French speakers often refer to the TOC as the index.

I might have been able to narrow the issue down. If the original NCX TOC contains invalid hrefs, the standardized EPUB will also contain invalid hrefs.

To test this, download the test epub that I attached to my previous post and do the following:

1. Open the epub.
2. Manually change the following line in the ncx file:

from:

Code:

<content src="txt/Scheer%2CK.-H.%26Darlton%2CClark-%5BPerry%20Rhodan-001%5DOperation%20Astree%20%281961%29_split_006.xhtml"/>

to:

Code:

<content src="test/Scheer%2CK.-H.%26Darlton%2CClark-%5BPerry%20Rhodan-001%5DOperation%20Astree%20%281961%29_split_006.xhtml"/>

3. Select Restructure Epub to Sigil Norm.

The above entry will be changed to:

Code:

<content src="../test/Scheer%2cK.-H.%26Darlton%2cClark-%5bPerry%20Rhodan-001%5dOperation%20Astree%20%281961%29_split_006.xhtml"/>

Ideally, Sigil should make sure that all NCX file hrefs are valid and refuse to run Restructure Epub to Sigil Norm, if invalid hrefs were found.

D.

DiapDealer · 11-10-2019, 05:25 AM

Quote:

Originally Posted by BetterRed

Any known reason why I shouldn't try it in the Windows 10 Pro Sandbox ?

BR

None that I know of. Though I've absolutely no experience with W10's sandbox, so there's that to consider.

Notjohn · 11-10-2019, 06:38 AM

> French speakers often refer to the TOC as the index

Thank you, thank you! I am regularly puzzled by seeing that usage on the KDP forums, since an index, in American usage, always appears at the other end of the book (and has little practical benefit in an ebook).

najgori · 11-10-2019, 06:43 AM

Quote:

Originally Posted by DiapDealer

My god.

The ridiculousness of some filenames out there that need to be accommodated never ceases to amaze me.

Pluto & Verso publishing in order to protect their books from piracy change all filenames inside of epub to something like chapter01JohnSmithjoh43324nsmithgmailcom.xhtml where John Smith is buyers name and after that is buyers email address. They also add exlibris with buyers name written in crappy font with limited characters support.

books are really good, though.

DiapDealer · 11-10-2019, 07:59 AM

Quote:

Originally Posted by najgori

Pluto & Verso publishing in order to protect their books from piracy change all filenames inside of epub to something like chapter01JohnSmithjoh43324nsmithgmailcom.xhtml where John Smith is buyers name and after that is buyers email address. They also add exlibris with buyers name written in crappy font with limited characters support.

books are really good, though.

That's just silly. If there's DRM, such filenames are pointless. And if there's no DRM (regardless of how/why) the filenames are a mass rename away from being pointless. Either way: their efforts/money to thwart piracy would be better spent elsewhere.

If their books are drm-free, there's no court in the land that would prosecute based on filenames found in the wild, anyway. "Your honor: I misplaced my reader during that time and found it a week later at work. I have no idea who might have used it (or what they might have done with it) during that time." *shrug*

DiapDealer · 11-10-2019, 08:14 AM

Quote:

Originally Posted by un_pogaz

Also, I think that an option to automatically apply a structure (either Sigil's or the custom one) at the opening of any ePub, would be very useful (no ask user, old working mode).

Your bug report illustrates quite perfectly why such an automated feature could be very dangerous. That's not in the cards just now.

Quote:

Another good thing would be to be able to save several structures and be able to apply them easily (a "Structures" sub-menu).

We provide a way to define a new default structure. If someone wants multiple template structures to choose from, an input plugin would lend itself perfectly to that end. We're offering users a way to create alternative structures, to safely edit/save alternative structures, and convert an alternative structure to the traditional Sigil standard structure. Being able to quickly restructure to a wide range of alternative structures is probably not going to be an inherent part of Sigil. Input/Output plugins can be written to do that.

Since Doitsu is unable to recreate your filename issue with special characters, I can only assume it's not a special character problem at all. It sounds more like a broken or incorrect opf file. And that will always be a problem regardless of whether you're opening/saving the problem epub with an older version of Sigil, or using the Restructure tool with the latest version. GIGO still applies.

KevinH · 11-10-2019, 08:36 AM

Also you can use the BookBrowser "move" command to change the structure of one file or even an entire set of files of the same group to a new folder. With that and rename you can change the epub to any structure you want in just a few steps.

I agree with DiapDealer that multiple template structures would probably be better handled by a plugin.

KevinH · 11-10-2019, 08:40 AM

Hmm ... what gets deleted, the existing file, the file from the opf manifest or the toc entry?

If you could pm either DiapDealer or KevinH with a link to a copy of that epub for testing, that would be a big help.

If not, please load the epub, and then immediately run epubcheck in it and let us know what is broken. Also, pming or posting the opf, and ncx may help us to track the issue down.

Thanks for testing and reporting back!

KevinH

Quote:

Originally Posted by un_pogaz

Yeah, well done on the job.

Now bug report: First ePub, first bug :
I have applied a manual restructuring "OEBPS/Text/Scheer,K.-H.&Darlton,Clark-[Perry Rhodan-001]Operation Astree (1961)_split_006.html" is named in the OPF "../Scheer%2CK.-H.%26amp%3BDarlton%2CClark-%5BPerry%20Rhodan-001%5DOperation%20Astree%281961%29_split_006.html" (old index before restructuring)
Consequently, it is deleted.
I suspect a special character because the files in [A-z] only are correctly restructured.

Also, I think that an option to automatically apply a structure (either Sigil's or the custom one) at the opening of any ePub, would be very useful (no ask user, old working mode).

Another good thing would be to be able to save several structures and be able to apply them easily (a "Structures" sub-menu).

PS: I don't know if you planned to add it to this version, but fortunately Sigil warns us when he encounters a none (or poorly) indexed files.

EDIT: the files with special character are not even renamed, under any circumstances!
EDIT2 : It's even more vicious, if I save the ePub even without renaming anything, the second opening will fail.

un_pogaz · 11-10-2019, 08:52 AM

0) No Sandbox
1) This ePub is not mine

1.5) I agree that such a name is absurd
2) I opened and saved with 0.9.18 and then used 0.9.991 without any problem. It is therefore a problem when loading the OPF into Sigil 0.9.991.
Cef. OPFs.

Apparently it is the & ampsand that is treated differently. It's parse in XML format before then in "web url" format au moment du chargement.

Source

Quote:

0.9.18 (work)

Quote:

0.9.991

Quote:

un_pogaz · 11-10-2019, 08:57 AM

And HTML is deleted because it is not indexed.

(And I did a test with an empty NCX and same problem)

EDIT : Arf. I don't feel like I'm being clear (even for me) and now that I have the 3 <item> the problem is obvious to me.

In the original ePub, the file name reference is in XML format. At Loading it is translated directly in "web url", transforming the & into %26amp%3B instead of %26, which breaks the reference of the object.

0.9.18 seems to detect the format correctly and therefore avoids parse error.

11-09-2019, 10:08 PM	#1
DiapDealer Grand Sorcerer Posts: 27,552 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Sigil-0.9.991 (1.0 Pre-release) *Announcing Sigil-0.9.991 (pre Sigil-1.0) Alpha* Release** ---------------------------------------------------------------------------------- Before Sigil 1.0 can be released, we felt it was very important to remove the restriction that all epubs must be in Sigil standard form which included one OEBPS folder, specific names for the OPF, and NCX, only one folder for each type of resource (Text, Images, etc.). Therefore, over the last two months, we have had to tear out much of the internals of Sigil to teach it to not standardize epubs on initial load, not to rename the OPF, NCX, not to move files to our specific folders, etc. Then we had to teach Sigil how to properly work with files in any structure or locations, how to build the correct links between them. This included extending the Sigil plugin interface appropriately. To accomplish this over 4000 lines of code were added and over 2000 were deleted, touching almost every file in Sigil. All of this work has been very disruptive with the potential for lots of breakage and new bugs being introduced. Special thanks must be given to both BeckyEbook and Doitsu for regularly building and testing the changes as they were being made and reporting back any breakage or issues they found and providing feedback on ideas. Without their hard volunteer effort, Sigil would be in a much much worse condition. The work on Sigil-1.0 has now stabilized, and it is now ready for additional volunteers and testers to try and break it so that any final nits or issues can be tracked down and fixed before the upcoming Sigil 1.0 release. The main changes include the following: Upon initial load, the epub structure that exists will be loaded and kept in its "as-is" state, moving and renaming nothing The BookBrowser will still show the Text, Images, Styles, Fonts, Audio, Video, and Misc "folders" but these are now "virtual folders" that simply organize the existing resources by type but do not restrict their actual location within the epub. A new Preferences setting has been added to the Appearance Preferences that will determine if the book folder relative path will be shown in the BookBrowser or its shorter unique name (filename) will be used instead. In either case, the BookBrowser tooltips will always show the file's true book folder relative path. Duplicate file names (in different subfolders) will always show the longer path to be able to differentiate between them regardless of the new Appearance Preference. A Tools menu item has been added to standardize any epub already loaded to Sigil's old structure A Tools menu item has been added to allow a user to design a custom "empty" epub layout A "Move" context menu item has been added to BookBrowser to enable moving of resources to where you want them to be. A "Rename" context menu item has been added to BookBrowser to enable renaming of the NCX and OPF resources. The python plugin interface has been extended to support all of this as documented in the new Sigil_Plugin_Framework_rev12.epub and online at MobileRead in our Plugin Development thread. Additional bug fixes include: handle javascript location.href style in Preview use CaseInsensitive sort in Plugins menu (Thank you BeckDTP) use Locale aware sort routines for AddMetadata, and MetaData Names (Thankyou Doitsu) recognize x-font-otf style media types recognize vnd.adobe.page-template+xml media type recognize webp images media type improve updating of smil xml files when moved preserve whitespace inside the code element when prettifying try again to prevent timer events during MainWindow close that causes random crashes on exit on macOS fix svg cover images to use 100vh and 100vx in Preview Please note, the design goal of this project was to make minimal changes for users and ebook developers that still want to use Sigil as they did. The new Preference setting defaults to showing only the filename as before, and the default empty epub for creating new epubs is the same as before. The interface and functionality should stay almost exactly the same. Loading any epub previously saved by Sigil will still work exactly as before. Only editing new epubs that are in non-standard form will necessitate a use of the new Tool menu (see #4 above) to standardize it. Call For Feedback, Testing, Plugin Developers, etc... ---------------------------------------------------------------------------------- So we really need your help to test Sigil to make sure we have not broken anything and to provide feedback on the new features. This release will also serve to allow Plugin Developers to test their plugins and make any changes needed to make sure their plugins will still work as expected. So please take a minute to download Sigil-0.9.991 and test it as you would use earlier versions of Sigil, reporting back on MobileRead or via our GitHub site any bugs you find with instructions on how to recreate them. This is especially true of some of Sigil's more advanced features such as HTML TOC, Index generation, Merging and Splitting of Files, Reports, movement of resources, and any other feature of Sigil you would normally use on a regular basis. Though we're not going to ignore any reported issues, we're asking that you try to limit your reports to broken functionality or incorrect behavior. Cosmetic issues and minor inconveniences (with easy workarounds) can be dealt with later. Note: Sigil-0.9.991 should not be considered a "production-ready" release. It is being made available to make sure our upcoming Sigil 1.0 release is as bug free and stable as possible. Sigil-0.9.991 (1.0 pre-release) page and downloads.

11-10-2019, 02:32 AM	#2
un_pogaz Chalut o/ Posts: 410 Karma: 145324 Join Date: Dec 2017 Device: Kobo	Yeah, well done on the job. Now bug report: First ePub, first bug : I have applied a manual restructuring "OEBPS/Text/Scheer,K.-H.&Darlton,Clark-[Perry Rhodan-001]Operation Astree (1961)_split_006.html" is named in the OPF "../Scheer%2CK.-H.%26amp%3BDarlton%2CClark-%5BPerry%20Rhodan-001%5DOperation%20Astree%281961%29_split_006.html" (old index before restructuring) Consequently, it is deleted. I suspect a special character because the files in [A-z] only are correctly restructured. Also, I think that an option to automatically apply a structure (either Sigil's or the custom one) at the opening of any ePub, would be very useful (no ask user, old working mode). Another good thing would be to be able to save several structures and be able to apply them easily (a "Structures" sub-menu). PS: I don't know if you planned to add it to this version, but fortunately Sigil warns us when he encounters a none (or poorly) indexed files. EDIT: the files with special character are not even renamed, under any circumstances! EDIT2 : It's even more vicious, if I save the ePub even without renaming anything, the second opening will fail. Last edited by un_pogaz; 11-10-2019 at 02:49 AM.

11-10-2019, 08:57 AM	#15
un_pogaz Chalut o/ Posts: 410 Karma: 145324 Join Date: Dec 2017 Device: Kobo	And HTML is deleted because it is not indexed. (And I did a test with an empty NCX and same problem) EDIT : Arf. I don't feel like I'm being clear (even for me) and now that I have the 3 <item> the problem is obvious to me. In the original ePub, the file name reference is in XML format. At Loading it is translated directly in "web url", transforming the & into %26amp%3B instead of %26, which breaks the reference of the object. 0.9.18 seems to detect the format correctly and therefore avoids parse error. Last edited by un_pogaz; 11-10-2019 at 09:21 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
5.8.11 Pre=release	knc1	Kindle Developer's Corner	21	04-17-2018 08:42 PM
Overdrive - Search for Pre-Release	nynaevelan	General Discussions	11	11-24-2013 02:27 PM
Where are the pre-release purchase buttons?	Sydney's Mom	General Discussions	8	09-06-2012 10:57 PM
KF Android 4.0 Pre-Release Version	robertc88	Kindle Fire	22	01-22-2012 07:24 PM
PDF Viewer 0.3.0 pre-release	pruss	Android Devices	62	11-22-2011 11:18 AM

11-10-2019, 04:29 AM	#3
BetterRed null operator (he/him) Posts: 20,575 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	Any known reason why I shouldn't try it in the Windows 10 Pro Sandbox ? BR

11-10-2019, 06:38 AM	#8
Notjohn mostly an observer Posts: 1,515 Karma: 987654 Join Date: Dec 2012 Device: Kindle	> French speakers often refer to the TOC as the index Thank you, thank you! I am regularly puzzled by seeing that usage on the KDP forums, since an index, in American usage, always appears at the other end of the book (and has little practical benefit in an ebook).

11-10-2019, 08:36 AM	#12
KevinH Sigil Developer Posts: 7,647 Karma: 5433388 Join Date: Nov 2009 Device: many	Also you can use the BookBrowser "move" command to change the structure of one file or even an entire set of files of the same group to a new folder. With that and rename you can change the epub to any structure you want in just a few steps. I agree with DiapDealer that multiple template structures would probably be better handled by a plugin.

Advert

Advert