How to make Sigil live (in calibre) - Page 13

eschwartz · 12-04-2013, 04:45 PM

Well, Sigil warns you then asks if you want it fixed automatically, or do it yourself. That is the right approach.

Giving the user a choice is key.

arspr · 12-04-2013, 05:00 PM

Quote:

Originally Posted by kovidgoyal

as always, patches are welcome.

I suppose it is really hard for coders seeing how a lot of people (like myself) ask for features without offering any REAL contribution... (but donations in the best occasions).

I suppose John Schember is not really happy with what has happened with his project...

I insist: thank you very much and congratulations for your GREAT effort.

eschwartz · 12-04-2013, 05:02 PM

Quote:

Originally Posted by arspr

I suppose it is really hard for coders seeing how a lot of people (like myself) ask for features without offering any REAL contribution... (but donations in the best occasions).

I suppose John Schember is not really happy with what has happened with his project...

I insist: thank you very much and congratulations for your GREAT effort.

Ah, but in a few years when I've learned a little more about coding I will be patching, so it's not all bad news.

kovidgoyal · 12-04-2013, 09:33 PM

Automatic correction of invalid html is not going to change. It is the foundation of many robust features in Tweak Book. I find the entire idea of software that refuses to render html because it is not valid XML, ridiculous, it is simply too easy to generate non well formed XML. Fortunately, the larger web development community agrees with me, see HTML5. If ebooks had developed just a few years later, epub would have been based on HTML 5 instead of XHTML and everyone's life would have been a lot better. Remember that Tweak Book is not a epub 2 specific tool, unlike Sigil. Therefore, its fundamental design choices are not going to be limited by epub 2.

And note that auto-correction in Tweak Book is based on the HTML 5 parsing algorithm, therefore the result is exactly what you would get if you viewed the HTML in a browser (and ebook readers are all going to be based on browser engines in the future (indeed most of them already are)).

@arspr: Yes, like DTD, xml declarations are another pointless legacy appendage that serve no useful purpose other than specifying character encoding.

And I dont mind when people request features, as long as they in turn dont mind if I refuse to implement some of them. The thing that annoys me (and I suspect most developers) is an attitude that a few users have, that their feature request *must* be implemented or that the developer owes them some kind of detailed explanation as to why the feature wont be implemented.

arspr · 12-05-2013, 04:34 PM

Quote:

Originally Posted by kovidgoyal

Automatic correction of invalid html is not going to change. It is the foundation of many robust features in Tweak Book. I find the entire idea of software that refuses to render html because it is not valid XML, ridiculous, it is simply too easy to generate non well formed XML. Fortunately, the larger web development community agrees with me, see HTML5. If ebooks had developed just a few years later, epub would have been based on HTML 5 instead of XHTML and everyone's life would have been a lot better. Remember that Tweak Book is not a epub 2 specific tool, unlike Sigil. Therefore, its fundamental design choices are not going to be limited by epub 2.

And note that auto-correction in Tweak Book is based on the HTML 5 parsing algorithm, therefore the result is exactly what you would get if you viewed the HTML in a browser (and ebook readers are all going to be based on browser engines in the future (indeed most of them already are)).

I still don't agree with you in this point but, hey, you are the coder and I'm just enjoying your work... I'll try to post my point of view once more and then if I do not convince you (probably because your criterion is far more appropriate than mine) I'll shut up forever on this issue

I agree that the automatic tools like Fix HTML, Beautify, Format Converter or even just the default book viewer MUST be fully silent and smart against this kind of syntax errors, solving them on the fly in the best possible way.

But if you are manually tweaking an ebook (epub, azw3, html5 or html234, it doesn't matter), then you are supposed to have some knowledge about what you are doing. And then what you do need is being told that you've made a mistake and where you've made it. You should always have the option to fix it yourself, because the correct fix isn't probably the automated one (as my small previous example showed). Of course, if you really want, you can always perform an automated and silent "Fix HTML" procedure. But I really think that being oblivious of your own bugs because the advanced tweaker doesn't give any kind of feedback is not a "serious" option in a "developer" tool.

And I don't really have any preference in HOW I'm given the warning by the software. Sigil uses a warning text and a rendering stop in that point on the book preview window, but other systems could be more desirable (or not). I just want some kind of advice/warning/whatever over an automatic (and possibly wrong) fix, which I know I can always launch nevertheless.

Now a possible bug. Look at the screenshot of a real CSS. (Forgive my joke with the screenshot name):

font-size in line 80 is violet but in line 106 is green
After the font-size in 80 you have ":" in blue italics. In every other place they are violet. The value "small" is also blue. And the ending semicolon is also different.
Look at the multi coloured page-break-before in line 97

EDITED

And another possible bug. Look at the other screenshot which shows the default Calibre Viewer and the File Viewer included in the tool.

The embedded font is not being used. CharisWhatever.ttf.
The "A Fire Upon The Deep" paragraph has a page-break-before: always; which is also being ignored. Look at the inspector.

user_none · 12-05-2013, 07:17 PM

Quote:

Originally Posted by arspr

I suppose it is really hard for coders seeing how a lot of people (like myself) ask for features without offering any REAL contribution... (but donations in the best occasions).

I suppose John Schember is not really happy with what has happened with his project...

Quote:

Originally Posted by kovidgoyal

And I dont mind when people request features, as long as they in turn dont mind if I refuse to implement some of them. The thing that annoys me (and I suspect most developers) is an attitude that a few users have, that their feature request *must* be implemented or that the developer owes them some kind of detailed explanation as to why the feature wont be implemented.

That's precisely the issue. Asking for new features is welcome, demanding new features and becoming abusive when they aren't implemented as demanded within a day or two is not welcome.

@kovidgoyal, please let me know when you feel tweak-epub's Sigil like feature integration is feature complete and stable. I'll make an announcement on the Sigil blog saying this is the functional and spiritual successor.

Katsunami · 12-05-2013, 07:41 PM

Maybe a thought, for people who are looking to use something like Sigil, but don't need or want the entire Calibre suite: are you going to rig something up in the installer to only install the ebook-tweak executable and the parts it needs, or are the needs of ebook-tweak so interwoven with Calibre itself that you'd end up installing everything anyway?

kovidgoyal · 12-05-2013, 10:40 PM

@arspr: As is mentioned in the welcome text on the preview screen, page breaks, page margins and embedded fonts whose font-family names in the css dont match the font-family names inside the actual font will not work in the preview.

And I'm afraid I am never going to agree that preventing all the automatic tools, the live preview and the live css editor (I am currently working on) from working just because you have forgotten to close a tag is a good idea. Imagine you are editing an HTML file and you decide to enter a new paragraph, you type in the <p> and start typing in text, everything will stop working until you type the closing </p>. That is just silly.

And the calibre philosophy has always been to empower people to do things even if they dont have as much technical knowledge as you or I. Having an editor that punishes you in scenarios like the previous one, is simply not compatible with my philosophy.

As it it, if you feel you want to have exact control over the HTML in the file, just never use any of the automated tools, or make sure that all the HTML is always perfectly valid XML before running the automated tools, then nothing will be changed apart from the doctype, the xml declaration and entities. Everything else will be exactly as you typed it.

I am done talking on this subject. As always, if you disagree, you are welcome to your own opinions, just realize that I am just as entitled to mine. And since I am doing all the work, my opinion prevails.

kovidgoyal · 12-05-2013, 10:46 PM

@Katsunami: I have absolutely zero interest in making a standalone installer for ebook-tweak. There already exists a standalone executable for it within calibre. If there are people silly enough to refuse to use ebook-tweak because they have to *install* the entire calibre package, that is their loss.

@user_none: Will do. It will be a while before ebook-tweak reaches feature parity with Sigil. I will be making a public release of ebook-tweak in a week or two as I think it is useable and useful already. I have never used Sigil myself, but I think the major bits that are missing are spell check, clips, error checking and a book view editor (which likely will never get implemented)

@arspr: CSS syntax highlighting highlights CSS keywords like small and numbers differently. That is by design. The multicolored page-break-before, incorrect highlighting after comment inside rule and semi-colons after percentages do look like bugs, I will look into those.

kovidgoyal · 12-05-2013, 11:22 PM

Today's release boasts an all new image editor, that allows you to trim borders, resize, rotate and apply various automated filters like sharpen, blur, de-speckle and so on. See attached screenshot. Should be useful for quick/simple touchups of images in the book without needing to use an external image editor. Of course for any major work, you will need ot use a proper image editor.

eschwartz · 12-05-2013, 11:28 PM

It's nice that we can finally see what images we have.

Couldn't care less about EDITING them, although I'm sure lots of people will find it useful.

Hitch · 12-06-2013, 01:32 AM

Quote:

Originally Posted by kovidgoyal

Automatic correction of invalid html is not going to change. It is the foundation of many robust features in Tweak Book. I find the entire idea of software that refuses to render html because it is not valid XML, ridiculous, it is simply too easy to generate non well formed XML. Fortunately, the larger web development community agrees with me, see HTML5. If ebooks had developed just a few years later, epub would have been based on HTML 5 instead of XHTML and everyone's life would have been a lot better. Remember that Tweak Book is not a epub 2 specific tool, unlike Sigil. Therefore, its fundamental design choices are not going to be limited by epub 2.

Kovid:

I know you said you were through discussing this, but my sole comment, if I may, is that having Tweak "auto-fix" HTML, as it does, gives the creator no audit trail. Using the example provided, without proofing the entire book, you wouldn't find that the closing italics tag had been moved to the wrong place. From my own standpoint solely (understanding quite clearly that my needs are very different from most here), this would never work for us. Is there any possibility that at least an audit trail of the auto-fixes could be somehow visible? e.g., a "fixed this" report of some kind?

For anyone making other people's books, using pretty cruddy manuscripts, this could be sort of death-defying. I understand your philosophy: Calibre helps those who cannot help themselves. And, Calibre is not really designed for "my ilk." But as this is intended to be a successor to Sigil, it would be great if at least the changes made were not silent, somehow.

Is there a way that that could be done? Which would replace the need for the "stop rendering" methodology that Sigil currently uses?

(Just asking.)

Hitch

kovidgoyal · 12-06-2013, 02:13 AM

@Hitch: Again, the only time HTML is automatically corrected is if you run one of the automated tools, like smarten punctuation or Fix HTML or rename files and so on. This has to be done, because the automated tools have to be able to parse and modify the HTML, which means auto-fixing it.

If all you do is open an html file, edit it, click save, the only automated change that happens is conversion of HTML entities to their unicode characters. The HTML itself is not touched. Therefore, if you want to make sure that no automated changes happen, simply ensure all your HTML is parseable XML before running any automated tool.

Finally, there will be an error check tool that will tell you about unparseable XML in the book, along with other useful (as opposed to the junk produced by epubcheck) errors.

Divingduck · 12-06-2013, 02:42 AM

Kovid, the new image editor is really nice and useful. Thanks for this.

Unbelievable, what a fast progress this project is making.

One thing I am missing there is an overview to all pictures as i.e. a thump nail list. Then it will be a bit easier to find the picture I am looking to place in a text or to have an overview about what is in the file from a visual view. Maybe there is some when time to implement something like this.

Alternatively, as a report for the entire eBook like Sigils report (what in my view is one of the best extensions of Sigil if I like to know what is going on in an eBook). Anyway, I will dream a bit on this. I guess this is fare out of the scope and there are other things what are more important

kovidgoyal · 12-06-2013, 02:57 AM

@Divingduck: Certainly, a report tool or a thumbnail overview would be useful. However, I do want to finish up the basic functionality first before moving on to those sorts of things.

One of the pieces of basic functionality I intend to finish is a plugin system. Then people can (relatively) easily contribute useful tools as plugins.

12-05-2013, 10:46 PM	#189
kovidgoyal creator of calibre Posts: 45,367 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@Katsunami: I have absolutely zero interest in making a standalone installer for ebook-tweak. There already exists a standalone executable for it within calibre. If there are people silly enough to refuse to use ebook-tweak because they have to install the entire calibre package, that is their loss. @user_none: Will do. It will be a while before ebook-tweak reaches feature parity with Sigil. I will be making a public release of ebook-tweak in a week or two as I think it is useable and useful already. I have never used Sigil myself, but I think the major bits that are missing are spell check, clips, error checking and a book view editor (which likely will never get implemented) @arspr: CSS syntax highlighting highlights CSS keywords like small and numbers differently. That is by design. The multicolored page-break-before, incorrect highlighting after comment inside rule and semi-colons after percentages do look like bugs, I will look into those. Last edited by kovidgoyal; 12-06-2013 at 12:17 AM.

12-05-2013, 11:22 PM	#190
kovidgoyal creator of calibre Posts: 45,367 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Today's release boasts an all new image editor, that allows you to trim borders, resize, rotate and apply various automated filters like sharpen, blur, de-speckle and so on. See attached screenshot. Should be useful for quick/simple touchups of images in the book without needing to use an external image editor. Of course for any major work, you will need ot use a proper image editor. Attached Thumbnails

12-06-2013, 02:13 AM	#193
kovidgoyal creator of calibre Posts: 45,367 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@Hitch: Again, the only time HTML is automatically corrected is if you run one of the automated tools, like smarten punctuation or Fix HTML or rename files and so on. This has to be done, because the automated tools have to be able to parse and modify the HTML, which means auto-fixing it. If all you do is open an html file, edit it, click save, the only automated change that happens is conversion of HTML entities to their unicode characters. The HTML itself is not touched. Therefore, if you want to make sure that no automated changes happen, simply ensure all your HTML is parseable XML before running any automated tool. Finally, there will be an error check tool that will tell you about unparseable XML in the book, along with other useful (as opposed to the junk produced by epubcheck) errors. Last edited by kovidgoyal; 12-06-2013 at 02:19 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How to make Sigil live	varlog	Sigil	136	10-25-2013 02:34 PM
how many tocs does Sigil make?	Gregg Bell	Sigil	17	01-31-2013 09:39 AM
How do I make all .pdf's in my calibre library open in calibre instead of preview	morbo3000	Library Management	9	07-30-2011 10:39 AM
using sigil to make changes befote conversion	alansplace	Sigil	11	06-07-2011 07:50 AM
i can't make sigil run in fedora 14	nastarovia	Sigil	1	01-20-2011 05:02 AM

12-04-2013, 04:45 PM	#181
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Well, Sigil warns you then asks if you want it fixed automatically, or do it yourself. That is the right approach. Giving the user a choice is key.

12-04-2013, 09:33 PM	#184
kovidgoyal creator of calibre Posts: 45,367 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Automatic correction of invalid html is not going to change. It is the foundation of many robust features in Tweak Book. I find the entire idea of software that refuses to render html because it is not valid XML, ridiculous, it is simply too easy to generate non well formed XML. Fortunately, the larger web development community agrees with me, see HTML5. If ebooks had developed just a few years later, epub would have been based on HTML 5 instead of XHTML and everyone's life would have been a lot better. Remember that Tweak Book is not a epub 2 specific tool, unlike Sigil. Therefore, its fundamental design choices are not going to be limited by epub 2. And note that auto-correction in Tweak Book is based on the HTML 5 parsing algorithm, therefore the result is exactly what you would get if you viewed the HTML in a browser (and ebook readers are all going to be based on browser engines in the future (indeed most of them already are)). @arspr: Yes, like DTD, xml declarations are another pointless legacy appendage that serve no useful purpose other than specifying character encoding. And I dont mind when people request features, as long as they in turn dont mind if I refuse to implement some of them. The thing that annoys me (and I suspect most developers) is an attitude that a few users have, that their feature request must be implemented or that the developer owes them some kind of detailed explanation as to why the feature wont be implemented.

12-05-2013, 07:41 PM	#187
Katsunami Grand Sorcerer Posts: 6,111 Karma: 34000001 Join Date: Mar 2008 Device: KPW1, KA1	Maybe a thought, for people who are looking to use something like Sigil, but don't need or want the entire Calibre suite: are you going to rig something up in the installer to only install the ebook-tweak executable and the parts it needs, or are the needs of ebook-tweak so interwoven with Calibre itself that you'd end up installing everything anyway?

12-05-2013, 10:40 PM	#188
kovidgoyal creator of calibre Posts: 45,367 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@arspr: As is mentioned in the welcome text on the preview screen, page breaks, page margins and embedded fonts whose font-family names in the css dont match the font-family names inside the actual font will not work in the preview. And I'm afraid I am never going to agree that preventing all the automatic tools, the live preview and the live css editor (I am currently working on) from working just because you have forgotten to close a tag is a good idea. Imagine you are editing an HTML file and you decide to enter a new paragraph, you type in the <p> and start typing in text, everything will stop working until you type the closing </p>. That is just silly. And the calibre philosophy has always been to empower people to do things even if they dont have as much technical knowledge as you or I. Having an editor that punishes you in scenarios like the previous one, is simply not compatible with my philosophy. As it it, if you feel you want to have exact control over the HTML in the file, just never use any of the automated tools, or make sure that all the HTML is always perfectly valid XML before running the automated tools, then nothing will be changed apart from the doctype, the xml declaration and entities. Everything else will be exactly as you typed it. I am done talking on this subject. As always, if you disagree, you are welcome to your own opinions, just realize that I am just as entitled to mine. And since I am doing all the work, my opinion prevails.

12-05-2013, 11:28 PM	#191
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	It's nice that we can finally see what images we have. Couldn't care less about EDITING them, although I'm sure lots of people will find it useful.

12-06-2013, 02:42 AM	#194
Divingduck Wizard Posts: 1,166 Karma: 1410083 Join Date: Nov 2010 Location: Germany Device: Sony PRS-650	Kovid, the new image editor is really nice and useful. Thanks for this. Unbelievable, what a fast progress this project is making. One thing I am missing there is an overview to all pictures as i.e. a thump nail list. Then it will be a bit easier to find the picture I am looking to place in a text or to have an overview about what is in the file from a visual view. Maybe there is some when time to implement something like this. Alternatively, as a report for the entire eBook like Sigils report (what in my view is one of the best extensions of Sigil if I like to know what is going on in an eBook). Anyway, I will dream a bit on this. I guess this is fare out of the scope and there are other things what are more important

12-06-2013, 02:57 AM	#195
kovidgoyal creator of calibre Posts: 45,367 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@Divingduck: Certainly, a report tool or a thumbnail overview would be useful. However, I do want to finish up the basic functionality first before moving on to those sorts of things. One of the pieces of basic functionality I intend to finish is a plugin system. Then people can (relatively) easily contribute useful tools as plugins.