Future Thoughts on Getting Decent epub 3 support

[KevinH]

Hi All,

DiapDealer and I have discussed how we will move forward. The following is a very tentative plan for getting to the point of some reasonable support of epub3 without breaking epub2 support in Sigil. You can think of it as a very rough roadmap to changes in Sigil after our Sigil-0.9.0 release.

All comments welcome ...

1. add support to Gumbo to always use nbsp / x160 entities depending on epub version being used

2. pull pretty print out of the Clean menus and create a prettyprint function or plugin

3. create an new metadata gui to allow editing both epub2 and epub3 metadata

4. add a nav generator (similar to the toc generator). Note: since an ncx ToC is allowed, Sigil will still create and maintain one even for epub3. A user preference for epub3 will be created to strip it out when saving

5. try to add support for mathml to preview/bookview either via the javascript mathml support or via our own Qt version.

6. try to add support for smil, and other supportive technologies

7. update to python 3.5.X at some point

8. possibly remove BookView completely at this point since it seems to cause more issues than it is worth (no firm decision on this one way or the other)

9. possibly remove support for Python 2.7 plugins after helping to convert all current ones over to run on both Python 2.7 and Python 3.4 (no firm decision on this one way or the other)

10. add support for running long python scripts internally to Sigil without slowing the ui

11. add buttons or keyboard accelerators to launch user configurable plugins with minimal gui overhead.

All comments welcome - ideas for other new features welcomed as well.

Hope this helps,

KevinH

[Arios]

8. Possibly remove BookView completely at this point since it seems to cause more issues than it is worth (no firm decision on this one way or the other)

It is an idea that has certainly gone through many heads!

Without denying the fact that BV reassures many new users (and can be useful sometimes, but also misleading...), I think the first crucial decision to take in the future development of Sigil is: what can simplify software maintenance ? And if some features like BV have to be sacrificed to achieve this result, it is a good tradeoff.

After all, older versions will always have this feature and the Preview pane is working.

In short, imho, I would put option 8 first.

[DiapDealer]

I feely admit that I see BV as a huge hurdle that is continually in the way of achieving things (development-wise) that would be relatively simple (or at least much simpler) otherwise. A not inconsiderable portion of Sigil's quirks and inconsistencies stem from the need to compromise in order to maintain the ability to edit in both Book View and Code View.

On the other hand, I'm not entirely unsympathetic to those who likely won't want to use Sigil without it, either. I like to think we would make every effort to avoid making that hard decision without leaving a relatively solid version of Sigil that those users could camp out with as long as they liked.

[KevinH]

Hi,

Originally Posted by Arios

what can simplify software maintenance ? And if some features like BV have to be sacrificed to achieve this result, it is a good tradeoff.

Good point. From a software maintenance point of view, removing Xerces, reducing the number of third party software pieces, removing boost, etc all helped immensely in that regard. Then making embedded python a build time tool (not part of the tree) instead of how it was originally planned, helped too.

Other things similar to that would be:

1. removing BookView
2. removing support for Python 2.7 plugins (after helping to port them)
3. removing prettyprint and making it a plugin

These would all simplify Sigil maintenance but at the same time, these all work now and could reasonably be kept working **if** they are important to users.

So I guess I am looking for guidance from users, which of these features are important to them, what we should keep, and which should go away to simplify things over the long run.

Thanks,

KevinH

[KevinH]

Originally Posted by DiapDealer

On the other hand, I'm not entirely unsympathetic to those who likely won't want to use Sigil without it, either. I like to think we would make every effort to avoid making that hard decision without leaving a relatively solid version of Sigil that those users could camp out with as long as they liked.

Yes, absolutely, When we can add features without removing any we keep things in master for each stable release. When (and IF) we remove major things, we create a new branch of parallel development but only after releasing a very stable version.

KevinH

[theducks]

I mostly use Preview (on a second Monitor)
Places where I use BV:
To find a file where I broke it with a Bulk S&R. You can not land on a broken file with BV. If you could figure out how to mark (red?) the files that are invalid (broken) in the file browser, that use would be gone.

Avoiding the quirky (fight you for location) with focus change between Preview frame and CV frame. (If you select the frame/scroll bar, PV syncs to CV when coming from CV. You must click the Text, but then that loses the CV location on you
Any way to prevent the CV from syncing except AFTER a initial Text area insertion point has been established. I am trying to say: clicking on the Scroll bar, sync will only be activated when an insert point has been established

[Arios]

To be honest, I use very often BV, for all sorts of reasons (for example, copy text without the codes, etc.) and if the Sigil facelift can be done without removing anything, it's great.

Presently we have the chance to have two available and very patient and open minded programmers, but I'm afraid that if the restructuring of Sigil is too long/complex, due of such nice heavy to bear features, those two individuals could get fed up, and retire.

Anyway, this new thread is truly relevant and it is hoped that many people give their opinion about essential functions to keep into Sigil.

[eschwartz]

Is there any need to remove support for python27 plugins?
Currently the launcher code seems to be working fine with polyglot python.

How much overhead is there really? is that slowing down the development?
It doesn't seem like Sigil would actually gain anything from removing it.


...


Getting rid of the issues surrounding Book View seems like a fairly worthy goal.
On the other hand, for many people Book View, even with all its issues, is still the main reason to stick with Sigil rather than trying calibre's Editor -- even for people who already have calibre installed.

So another factor that should be considered is, what niche does Sigil fill?

[RbnJrg]

I use BV practically in only one case, but a very important one to me: when I copy and paste text from web pages to Sigil; when that text is paste in BV, Sigil retains the format of that text. So, I'm not very sure of not wanting that feature any more

[theducks]

Originally Posted by RbnJrg

I use BV practically in only one case, but a very important one to me: when I copy and paste text from web pages to Sigil; when that text is paste in BV, Sigil retains the format of that text. So, I'm not very sure of not wanting that feature any more

Have you explored using 'View source (code)" in your WEB browser?

[KevinH]

Hi,

The real issue with Python 2.7 support is that I have to maintain the quite large patch set to make sigil_bs4 possible (make BS4 polyglot). In addition, python 2.7 will be a second class citizen as they would not have access to all of the site packages that the internal python 3 embedded one has.

So removing python 2.7 would reduce to my supporting just the new bs4 functions: prettyprint, serialize, decode_xml, and my bug fixes (which I hope to upstream). I could also remove compatibility_utils.py and a few if statements and try catch pieces required to handle python class name changes and the like.

In addition, I am also worried about supporting/dealing with/answering bug reports for python interpreters set-ups not under my control given the possibility of name collisions and things, how unicode is passed to the console, handling output streams , etc.

So all in all, removing python 2.7 would be useful, but I would only do that after helping all current plugins that are python 2.7 only become polyglot if we decided to go that way.

KevinH

Originally Posted by eschwartz

Is there any need to remove support for python27 plugins?
Currently the launcher code seems to be working fine with polyglot python.

How much overhead is there really? is that slowing down the development?
It doesn't seem like Sigil would actually gain anything from removing it.


...


Getting rid of the issues surrounding Book View seems like a fairly worthy goal.
On the other hand, for many people Book View, even with all its issues, is still the main reason to stick with Sigil rather than trying calibre's Editor -- even for people who already have calibre installed.

So another factor that should be considered is, what niche does Sigil fill?

[cybmole]

Sigil, as is, meets all my edit needs, and I use book and code views. Arguably , I could learn to use calibre editor instead but why. I like how sigil works, I like how it does some things by default that calibre editor does only when told to, and I like how it does not give me any original_fmt grief. (I put that last niche one in for eSchwartz 😁 )

[eschwartz]

Originally Posted by KevinH

Hi,

The real issue with Python 2.7 support is that I have to maintain the quite large patch set to make sigil_bs4 possible (make BS4 polyglot). In addition, python 2.7 will be a second class citizen as they would not have access to all of the site packages that the internal python 3 embedded one has.

So removing python 2.7 would reduce to my supporting just the new bs4 functions: prettyprint, serialize, decode_xml, and my bug fixes (which I hope to upstream). I could also remove compatibility_utils.py and a few if statements and try catch pieces required to handle python class name changes and the like.

In addition, I am also worried about supporting/dealing with/answering bug reports for python interpreters set-ups not under my control given the possibility of name collisions and things, how unicode is passed to the console, handling output streams , etc.

So all in all, removing python 2.7 would be useful, but I would only do that after helping all current plugins that are python 2.7 only become polyglot if we decided to go that way.

KevinH

Well, most plugins appear to be python3, and Sigil's internal routines don't need to worry about python27 compatibility.

One would already have to install python himself, and as well as any additional site-packages.
So anyone who wants to use a python27 plugin would have to install bs4 himself, as well.

[eschwartz]

Originally Posted by cybmole

Sigil, as is, meets all my edit needs, and I use book and code views. Arguably , I could learn to use calibre editor instead but why. I like how sigil works, I like how it does some things by default that calibre editor does only when told to, and I like how it does not give me any original_fmt grief. (I put that last niche one in for eSchwartz 😁 )

Your favorite hobby horse.

Sigil will give you just as much ORIGINAL_FMT grief as calibre's Editor will.

But then, you knew that already.

Don't bother responding -- one post is my new limit when it comes to your hobby horse.
And even that is simply because I don't want anyone else to mistakenly believe you.

[Toxaris]

Let's not get into the Calibre Editor vs Sigil discussion please. That is not relevant IMHO.

Some valid points have been made I think. I will update my own plugin(s) to ensure that they work on Python 3.4. Doitsu has sent me a message with most of the work already (thanks!!!). I feel, but I could be wrong, that removing the BookView would be a big improvement codewise. The issue is that some people are still using it. What I distill, most usecases for BV are:
- copy/paste from websites e.d. with retaining layout
- entering breaks (will correctly close and open tags, very useful for multiple tags)
- easier finding which file is broken

Now, for option 1 I assume a plugin could be written where content can be pasted and inserted in either a new file or at the place of the cursor (can be tricky since cursor position is not retrievable in the plugin environment AFAK). Option 2 is useful, but could be done manually of course. It is more convenience I think. For option 3 it might be useful that if a (x)html file cannot be rendered due to errors, the filename is made red in the book browser. Is that feasible?

I think that would take away most reasons that people are using BookView.