Saving/Loading times

Lemmy · 11-21-2020, 07:52 AM

I've just upgraded to Sigil 1.41 and find my currently in progress book takes something like 55 seconds to save, 30 seconds to load.

It's not a massive problem, though slightly disconcerting to see 'Not Responding' appear alongside the Sigil masthead and the screen go grey when saving. If left it does save, however.

This particular book is 20,000 words including about 35 pictures. My CSS and markup are minimalist. Similar previous books load in a few seconds and and I can't detect any great difference between them and my present one.

Any pointers why one ePub file, similar to another ostensibly, should take ten times as long to save? The fast loading file ePub is 179MB, this slow one 250MB.

My computer is a Win 10, Intel i7 with 32Gb RAM and all solid state disks.

The_book · 11-21-2020, 08:28 AM

How many html files and toc items?
Some days ago I tried to manage a dictionary in sigil but it just too long. That files contians 160mb html, with 160000 html files, which means opf file is over 20mb

KevinH · 11-21-2020, 08:50 AM

Is each chapter in its own file?

DiapDealer · 11-21-2020, 09:19 AM

Monolithic (x)HTML files are the only thing I've found that slows Sigil down. Granted: I wouldn't touch 100MB+ epub with a ten-foot pole, but still.

Lemmy · 11-21-2020, 01:31 PM

Quote:

Originally Posted by KevinH

Is each chapter in its own file?

Yes, 28 files/ 28 chapters. I have a series of books concerned with camera menus so each menu needs to be a separate item for ease of handling in both eBook and print form.

What is odd is that a 179MB file (25 chapters) saves in 3 or 4 seconds. This 250MB file takes nearly a minute. I haven't added the TOC yet but when I do this one will be about 140 items. The 170MB file I mentioned has 164.

Some of my original images in jpg form are 10MB. 5000px across which is oversized for the purpose but convenient of other uses. I suppose I could down-sample those to 1800px/300dpi for the books but I haven't needed to before, simply specifying sizes in width percentages in my CSS.

The actual ePub file seize is dictated by the subject matter so can't be cut. My CSS and markup are an absolute minimum so can't be altered.

DiapDealer · 11-21-2020, 01:46 PM

Files that have very few line-endings can slow things down considerably too, but a Mend, or better yet, Mend and Prettify should eliminate that issue. If your markup minimizations include the removal of line endings, that could have a drastic effect on Sigil performance.

KevinH · 11-21-2020, 01:47 PM

To speed up loading, you may want to turn off "Mend on Open" in Sigil preferences. But we still check for not well-formed html (even if you decide to not auto fix things) and that involves reading in each xhtml file and parsing it with a strict xml parser to look for errors.
This is what takes the most time even using multiple threads to do the testing.

For a future release, I will see if there is a way to easily prevent the well-formed check on initial load.

Lemmy · 11-21-2020, 03:13 PM

I don't really have any problem with bad HTML - a check rarely reveals any problems but I've been writing the stuff for a long time. Here's a typical page: can anyone see any problem?

Code:

++++++++++++++++++++++++
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops">
<head>
  <title></title>
<link href="../Styles/MenuBook.css" rel="stylesheet" type="text/css"/>
</head>

<body>
<h2>Custom Menu C1. Disp/(fig 34)/PC</h2>
<img alt="CustomC1Menu" class="figure" src="../Images/CustomC1Menu.jpg"/>
(fig 35) Control Settings
This sets what happens when you press the OK button while shooting. You can set it differently for each stills Mode, the choice being LV-C or LV-SCP. LV-C means Live Control. LV-SCP is the Super Control Panel.
<img alt="LV-CScreen" class="cameraback" src="../Images/LV-CScreen.jpg"/>
<div class="list">• LV-C: (top)gives you quick access to overall shoot settings like ISO or Aspect Ratio that you might want to alter during a shoot. You can select a parameter from the right of the screen using the up/down arrow keys or the rear dial and alter it with the right/left arrow keys or the front dial. The advantage of the LV-C screen is that it gives you a better of view of your image as you make adjustments. The disadvantage is that it is less comprehensive than the SCP.
</div>
<div class="list">• <img alt="LV-SCPScreen" class="cameraback" src="../Images/LV-SCPScreen.jpg"/>LV-SCP: (bottom) It gives you access to a much wider range of options such as Stabilization and the AF Area but is less quick to use. You can move around the screen using the rear dial or arrow pad or touching it if you prefer. Having highlighted a parameter, you can alter it with the rear dial, press OK or a double tap on it. For most photographers the SCP will be the quickest way to adjust settings sine it puts about everything you might want to alter all in one place, just a double tap away.
</div>
Note: there's no point in setting the SCP in P/A/S/M because in all those modes the (fig 36) button accesses it directly anyway.
<h3>(fig 37) /Info Settings</h3>
<h4>(fig 38) Info</h4>
This setting controls what happens when you press INFO while reviewing images. All unticked, the button does nothing. As you tick more boxes, a press on the INFO button when reviewing will step through them. The options are pretty straightforward.
<div class="list">• Overall gives you a histogram of the colour components of your image and the shooting information. 
</div>
<div class="list">• Highlights and Shadow gives you a flashing display of which picture areas are clipped, that is, do not contain detail, being beyond the camera's ability to capture them at the exposure used.
</div>
<div class="list">• Light Box gives you a way to compare two images side by side. Get the image you want on screen and press Info until a split screen comes up. On the left will be a section of the image you selected and on the right the next image taken. Scroll through your images using the left and right arrow keys or the front dial until you find the one you want to compare. Turning the rear dial will magnify both images.
</div>
<h4>LV-Info</h4>
This sets what you will see when you press the Info button while shooting pictures. With all boxes unticked toggles between an uncluttered screen and one with all the shooting information shown. You can add 1 or 2 custom screens to these, so that the Info button now toggles through 3 or 4 screens. The 2 screens offer the same 3 options on each. This is because some photographers might find the 3 extras all on one screen rather confusing. The options are the same for all screens.
<div class="list">• (fig 39): this is a histogram. It maps the brightness of pixels in an image from black on the left to white on the right and gives an indication of whether your exposure is optimal. The image Olympus use for their icon is a classically perfect histogram with no cut off of the 'mountain range' on dark or light sides. This represents an image with a dynamic range within the capabilities of the sensor. On a sunlit day, the difference in brightness between the deepest shadows and the brightest whites (the dynamic range) may be beyond the ability of the sensor to record it all. In this image (fig 40) you can see the histogram bleeds off to the left. That means that there are gradations of black that cannot be coped with by the sensor at this exposure and they will be reproduced as all uniform black. This may or may not matter, according to the amount of cut off and the detail contained in those pixels. You can bring those pixels within scope by increasing the exposure, which will move the entire range over to the right. There is a danger then that the brightest pixels will be out of the reproducible range(clipped). If so, the only option is a compromise to what seems to you the most attractive rendition.
</div>
<div class="list">• Highlights and Shadow - this gives you a graphic representation of the areas of your picture that will be beyond the dynamic range, the range of brightness, that the Olympus E-M10 MkIV's sensor can encompass. Areas that are too dark to be reproduced (underexposed) are coloured blue, too light (overexposed) are coloured red. Note: in all of this, it should be remembered that there is no 'correct' exposure artistically. Correct exposure technically means reproducing the image with the least amount of clipping possible. Artistically, you may prefer the look of an image which is technically under or overexposed. Let your eyes be the ultimate judge of what is right pictorially. It is part of what distinguishes the work of one photographer from another.
</div>
<div class="list">• Level Gauge - there is no quicker way to make an image look amateurish than tilting buildings or a wonky horizon. Olympus give you the chance to go straight with this option. It superimposes two bars on the screen which turn green when the image is straight horizontally and/ or vertically.</div>
<h4>(fig 37) Settings</h4>
If you go to Playback mode and turn the rear dial one click clockwise, you are presented with a grid view of all your images. This menu item lets you set how many images are presented to you at once plus, with further turns of the dial, which other screen you see (or not). Calendar lets you select a date and see what images you made that day.
<h3>Live View Boost</h3>
In normal shooting the camera gives you a view that matches what the recorded image will look like, including any exposure compensation applied. In very dim light, an image may be too dark to see properly. Set Live View Boost to On1 (bright) or On2 (brighter)and the camera gives you a screen view prioritizing easy viewing over accuracy. Note: what you see on screen may not match the recorded image and viewing may be jerky.
<h3>Flicker Reduction</h3>
Some forms of lighting induce a flickering effect on the EVF or monitor. Auto senses the problem and tries to lessen it. If that doesn't work, you can try the specific 50Hz and 60Hz settings. If you are in the USA, try 60Hz first, in Europe try 50Hz first. There's no reason to turn this Off, so Auto is the obvious setting.
<h3>Displayed Grid</h3>
This gives you a selection of grids that are overlaid on the screen. They don't appear on the recorded image, of course. These are handy for checking you are straight on to building, for example, or accurate placement of objects in a composition. Appropriately, the 3rd grid down provides you with the points for the Rule of Thirds or Golden Mean. Cliché, maybe, but it works.
<h3>Peaking Color</h3>
Choose the colour of edge outlining when using the Peaking focusing aid. The main thing is to use a colour that shows up clearly so you'll need to ring the changes as necessary.
</body>
</html>
++++++++++++++++++++++++++++++++++++++++++++++++++

DiapDealer · 11-21-2020, 04:09 PM

Quote:

Originally Posted by Lemmy

Here's a typical page: can anyone see any problem?

You mean other than almost all of the text being added directly to the body element instead of a proper div or p container?

DNSB · 11-21-2020, 04:38 PM

Quote:

Originally Posted by DiapDealer

You mean other than almost all of the text being added directly to the body element instead of a proper div or p container?

Rather oddly, epubcheck is happy with naked text in an epub3 while checking as an epub2 does seem to frown on naked text (Error while parsing file: text not allowed here; etc ) and img tags that are not wrapped in an element tag.

DiapDealer · 11-21-2020, 05:18 PM

Naked text is on my own personal "just don't do it. Ever" personal validation checklist. The fact that html5 might allow it doesn't make it a good idea, IMO.

How would you style any of it without styling ALL of it?

Not that I think it's relevant to slow loading times or anything.

Lemmy · 11-22-2020, 02:14 PM

Quote:

Originally Posted by DiapDealer

How would you style any of it without styling ALL of it?

Not that I think it's relevant to slow loading times or anything.

My books are complex in subject and to an extent layout, so the last thing I want to do is monkey around changing fonts and styling within the body! I am focused entirely on content and I think that is why my books sell so well. It is also why I use Sigil.

I tried exporting to ePub from Word and Google Docs and while both work, the resulting files are offensive in their bloat and complexity. HTML, CSS and Sigil are so straightforward to use that it's hard to see why anyone would use anything else (thank you. Mr Sigil).

I style the body tag and that is all I need. One way and another, my entire book needs only the 4 headings, 3 classes of image and body text.

Last - and most important, I have solved the speed problem. The images I use are mastered as tiff or psd in Photoshop and then exported as jpg to my image folder from where I do the import. I put all the images I will use into a folder on their own and then just import the lot to Sigil. I had left half a dozen large PSD files in the folder by mistake and then imported them. I found them neatly tucked away in Misc while exploring to find the problem. Sigil was spending some time working out what to do with them, I would guess. I'm rather impressed that it didn't just crash or stick on 'Not Responding', actually.

Thanks for your input, I appreciate it.

DiapDealer · 11-22-2020, 03:00 PM

To each their own. You asked if anyone saw problems. I mentioned what I thought were some. I would say you're limiting your ability to tweak your complex content layout by overzealously worrying about bloat. There's bloat, and then there's p tags.

Glad you figured out the load time issue. Thanks for reporting back on what the problem was.

Oh, and as one-half of the current "Mr. Sigil" ... you're welcome.

Lemmy · 11-22-2020, 03:33 PM

I take your point but in my own case my output is tailored to exactly what I need. I don't do anything for other people and I'd be happy enough pop in some <div> and <p> where necessary if I did.

I am a photographer by trade but used to write some software for my brother's business back in the 80s when disk space and run speed was actually a factor, both battered into submission by sheer power nowadays. But, old habits die hard and as with my photography, I get pleasure from removing what I find unnecessary.

I think I already donated but I'm over to do a bit more - thank you Mr Demi-Sigil 😁

DiapDealer · 11-22-2020, 05:40 PM

In all fairness (and with much respect), I'd say the title of Mr. Sigil should probably be reserved for its creator, Strahinja Marković.

11-21-2020, 07:52 AM	#1
Lemmy Junior Member Posts: 8 Karma: 10 Join Date: Nov 2016 Device: Kindle	Saving/Loading times I've just upgraded to Sigil 1.41 and find my currently in progress book takes something like 55 seconds to save, 30 seconds to load. It's not a massive problem, though slightly disconcerting to see 'Not Responding' appear alongside the Sigil masthead and the screen go grey when saving. If left it does save, however. This particular book is 20,000 words including about 35 pictures. My CSS and markup are minimalist. Similar previous books load in a few seconds and and I can't detect any great difference between them and my present one. Any pointers why one ePub file, similar to another ostensibly, should take ten times as long to save? The fast loading file ePub is 179MB, this slow one 250MB. My computer is a Win 10, Intel i7 with 32Gb RAM and all solid state disks.

11-21-2020, 01:46 PM	#6
DiapDealer Grand Sorcerer Posts: 28,358 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Files that have very few line-endings can slow things down considerably too, but a Mend, or better yet, Mend and Prettify should eliminate that issue. If your markup minimizations include the removal of line endings, that could have a drastic effect on Sigil performance. Last edited by DiapDealer; 11-21-2020 at 01:49 PM.

11-21-2020, 05:18 PM	#11
DiapDealer Grand Sorcerer Posts: 28,358 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Naked text is on my own personal "just don't do it. Ever" personal validation checklist. The fact that html5 might allow it doesn't make it a good idea, IMO. How would you style any of it without styling ALL of it? Not that I think it's relevant to slow loading times or anything. Last edited by DiapDealer; 11-21-2020 at 05:23 PM.

11-22-2020, 03:00 PM	#13
DiapDealer Grand Sorcerer Posts: 28,358 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	To each their own. You asked if anyone saw problems. I mentioned what I thought were some. I would say you're limiting your ability to tweak your complex content layout by overzealously worrying about bloat. There's bloat, and then there's p tags. Glad you figured out the load time issue. Thanks for reporting back on what the problem was. Oh, and as one-half of the current "Mr. Sigil" ... you're welcome. Last edited by DiapDealer; 11-22-2020 at 03:05 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Times of Destruction: Christian End Times Thriller	Cliff Ball	Self-Promotions by Authors and Publishers	0	02-15-2014 11:56 AM
PRS-T1 NY Times not loading	JethroJ	Sony Reader	3	10-01-2012 03:07 PM
PRS-950 Loading Times	anamardoll	Sony Reader	6	08-16-2011 09:35 AM
Losing files when saving saving to disk	theaccountant	Library Management	4	03-10-2011 02:38 PM
Journal Not Saving Writing When Moving to a New Page or Saving it	eberhardt333	enTourage Archive	5	11-24-2010 12:47 AM

11-21-2020, 08:28 AM	#2
The_book Zealot Posts: 100 Karma: 10 Join Date: Aug 2019 Device: none	How many html files and toc items? Some days ago I tried to manage a dictionary in sigil but it just too long. That files contians 160mb html, with 160000 html files, which means opf file is over 20mb

11-21-2020, 08:50 AM	#3
KevinH Sigil Developer Posts: 8,478 Karma: 5703586 Join Date: Nov 2009 Device: many	Is each chapter in its own file?

11-21-2020, 09:19 AM	#4
DiapDealer Grand Sorcerer Posts: 28,358 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Monolithic (x)HTML files are the only thing I've found that slows Sigil down. Granted: I wouldn't touch 100MB+ epub with a ten-foot pole, but still.

11-21-2020, 01:47 PM	#7
KevinH Sigil Developer Posts: 8,478 Karma: 5703586 Join Date: Nov 2009 Device: many	To speed up loading, you may want to turn off "Mend on Open" in Sigil preferences. But we still check for not well-formed html (even if you decide to not auto fix things) and that involves reading in each xhtml file and parsing it with a strict xml parser to look for errors. This is what takes the most time even using multiple threads to do the testing. For a future release, I will see if there is a way to easily prevent the well-formed check on initial load.

11-22-2020, 03:33 PM	#14
Lemmy Junior Member Posts: 8 Karma: 10 Join Date: Nov 2016 Device: Kindle	I take your point but in my own case my output is tailored to exactly what I need. I don't do anything for other people and I'd be happy enough pop in some <div> and <p> where necessary if I did. I am a photographer by trade but used to write some software for my brother's business back in the 80s when disk space and run speed was actually a factor, both battered into submission by sheer power nowadays. But, old habits die hard and as with my photography, I get pleasure from removing what I find unnecessary. I think I already donated but I'm over to do a bit more - thank you Mr Demi-Sigil 😁

11-22-2020, 05:40 PM	#15
DiapDealer Grand Sorcerer Posts: 28,358 Karma: 203720150 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	In all fairness (and with much respect), I'd say the title of Mr. Sigil should probably be reserved for its creator, Strahinja Marković.

Advert

Advert