Image editing creates unmanifested images

[AlanHK]

Often I find an epub with a huge image, and want to trim it down.

I routinely use IrfanView to do this.

It's convenient to r-click on the image in Sigil, choose "Open with" and then I am working on the image in a temp directory which when I exit IrfanView replaces the original image.

However, one tool in Irfanview is "Jpeg lossless crop", which is self explanatory.

The usual way to use it is select an area of the image, choose the option and your cropped file replaces the original.
It normally makes a backup of the original file in with "_org" added to the name.

When I exit Irfanview it seems OK, the new image is in place. However, I noticed that the filesize had increased, not decreased, though the new image is smaller.

F7 says "No problems found". If I close and reopen the epub, no errors are reported.

Eventually I do an epubcheck:

item 'OEBPS/Images/cover_org.jpg' exists in the EPUB but is not declared in the OPF manifest

And so I open the epub as a zip and see that I have both the new image cover.img and the backup of the old one: cover_org.jpg.

I can delete it from the zip and fix it, but aside from epubcheck, it's totally invisible in Sigil, not listed in the file tree.

Presumably, Irfanview renames the original file, makes the cropped one and names that as the original filename and Sigil is confused.

If you aren't suspicious you can end up with an epub bloated with hidden images.

[KevinH]

IrfanView should have a way to disable creation of unwanted backup _org files. Perhaps a command line switch? Have you asked on the IrFanView support site how to do that?

[AlanHK]

Originally Posted by KevinH

IrfanView should have a way to disable creation of unwanted backup _org files. Perhaps a command line switch? Have you asked on the IrFanView support site how to do that?

There is a way to do that,

EDIT-- actually, there isn't.



Of the three save options, only the second which use the original filename, and creates the backup, will be imported back to the epub. The other options will make a new file that Sigil ignores.

If I could easily find the temp folder these files are in I could delete the backup file. But the location isn't obvious and would take a while to find. This makes it all more hassle than extracting the file and editing and importing back to Sigil.

Now I'm aware of it I can delete any extra files using other tools.

Those who don't use EPUBchek though would have no hint that these backup files even existed in the epub.

I thought it deserved attention: why is Sigil importing the file at all, but not adding to the manifest? And could it happen in other cases with other programs launched by "open with" on image or html files?

-- PPS If I use Irfanview to simply rename the file, when I revert to Sigil, the original filename is still listed in the manifest, but is not there, and the renamed file is in the Images folder of the zip, but is invisible to Sigil. Both these are reported as errors by EPUBcheck. Of course, renaming the file like that is a stupid thing to do, but it illustrates the issue. I guess some operations use the file name; others a file handle.

[DiapDealer]

It adds the backup image to the archive because Irfanview is creating new file in Sigil's scratch area. The same thing would happen if you manually copied a file into the the epub's working temp folder while the epub was open for editing. Everything there is included in the zip archive when you save the epub. It's not manifested because it's being added to the archive at the filesystem level rather than with Sigil's API. Same for your rename scenario. The file is being renamed at the filesystem level by Irfanview, so Sigil has no idea whatsoever that the manifest needs updated.

It's a shame Irfanview has no other options to work with, because I don't see any easy way to fix it on Sigil's end. Not without hard-coding an Irfanview-specific hack, anyway. The only other possible solution would be to scan for, and delete, any unmanifested files found after an OpenWith operation completes. And that just sounds like overkill to me. And that still wouldn't account for any renaming of files with an external editor.

[Turtle91]

I use photoshop to edit my images and don't have any issues when I save with the same file name. The only issues I run up against is if I am changing the filetype, eg .jpg to .png

In that case, I use the save-as function - save to the desktop, and then use Sigil's "Add-Existing Files" function to add the new image.

[DiapDealer]

Originally Posted by Turtle91

I use photoshop to edit my images and don't have any issues when I save with the same file name. The only issues I run up against is if I am changing the filetype, eg .jpg to .png

In that case, I use the save-as function - save to the desktop, and then use Sigil's "Add-Existing Files" function to add the new image.

That's the only way it will work if you need to convert an image do a different format. Sigil's external editor Open-With feature is literally only designed to allow a single file to be edited and resaved. It has no capacity to deal with renamed, or newly-created files. Those need to be handled with integral Sigil functions and not external tools. Anything that would require an update to the epub's opf/ncx file (or any other file in the epub for that matter) cannot be done with an external editor without some degree of breakage.

[Turtle91]

You bet - I wasn't saying it's broken, just saying that's how I get around that issue. AlanHK (or anyone) could try the same technique to bypass the IrfanView issue.

[AlanHK]

Originally Posted by DiapDealer

Not without hard-coding an Irfanview-specific hack

It isn't that Irfanview is doing anything exceptional, it's because Sigil hands over its folder to another program and then zips it up again without checking if the contents still match the manifest. It's quite possible other programs could create their own backups or scratch files in the folder that get rolled into the epub

Originally Posted by DiapDealer

the only other possible solution would be to scan for, and delete, any unmanifested files found after an OpenWith operation completes. And that just sounds like overkill to me. And that still wouldn't account for any renaming of files with an external editor.

That seems like the correct thing to do rather than possibly putting large unmanifested files in the archive. An accidental steganography.

I'm not saying this for my own convenience; I can check and clean it up myself now that I know it can happen.

Originally Posted by Turtle91

You bet - I wasn't saying it's broken, just saying that's how I get around that issue. AlanHK (or anyone) could try the same technique to bypass the IrfanView issue.

I already mentioned that option in my second post.

[DiapDealer]

It sounds like a lot of trouble for a little-used feature (that even fewer run into trouble with) to me. You're the first to report an external editor adding an unmanifested file to an epub since the Open With feature was added back in 0.6.x or 0.7.x.

If we can come up with an easy way to prevent the problem, we surely will. But a considerable overhaul of the Open With feature code would quite frankly be more trouble than the feature is worth in my opinion.

[AlanHK]

Originally Posted by DiapDealer

You're the first to report an external editor adding an unmanifested file

That it hasn't been reported doesn't mean it hasn't happened.
There is no indication unless you have installed and use EPUBcheck.

How about having the built-in F7 check the manifest against the actual contents; and adding any extra files to it so they can be dealt with? And reporting any that are missing.

Anyway, I've made (or failed to make) my point. I won't hassle you any more.

[BetterRed]

@AlanHK - ask Irfan Skiljan to put in a fourth option to overwrite with same name with an option to save original elsewhere.

He did a couple of similar changes for me a few years back - i.e. additional options, for existing features.

'Bout time we had a new IV release

Meantime when you need to use the Lossless plugin,

• right click image Book Browser,
• select Save as, <whatever>.jpeg to let's say Desktop
• edit it there with IrfanView,
• save it (the original will be renamed as <filename>_org.jpeg),
• add <whatever>.jpeg back into Sigil via right click-on image folder, you can overrwrite existing <whatever>.jpeg

Or here's a list of programs that do lossless crop and rotate ==>> http://jpegclub.org/losslessapps.html

I would guess they all use jpegtran

BR

[BetterRed]

Originally Posted by KevinH

IrfanView should have a way to disable creation of unwanted backup _org files. Perhaps a command line switch? Have you asked on the IrFanView support site how to do that?

Lossless Rotate and Crop are done via a plugin, not sure that IV plugins are accessible from the command line.

BR

[AlanHK]

Originally Posted by BetterRed

Meantime when you need to use the Lossless plugin,

I know how to do that. That's how I did it before I noticed the "Open With".

-- Actually, simpler way:

Open image with Irfanview:
Do lossless crop
Type "T" for thumbails:
now you see thumbs of all the images in the folder. Select the _org backup and delete. Exit irfanview.
No extra file added to epub.



Anyway, it is not just Irfanview. Any program that creates a backup or other file in the same folder will cause these extra files to be included in the epub.

For instance, just selected an XHTML file, "opened with" Ultraedit, edited, saved. The edited file is fine.
Run EPUBcheck:

"... king-01.html.bak' exists in the EPUB but is not declared in the OPF manifest."


-- Now this is strange. Save epub. Close. Reopen.
Epubcheck: "No problems found!".

BUT: open epub as zip, the .bak is still there.


Sorry for beating the dead horse ... NOW I'll let it go.

[DiapDealer]

Originally Posted by AlanHK

How about having the built-in F7 check the manifest against the actual contents; and adding any extra files to it so they can be dealt with? And reporting any that are missing..

The well-formedness check (F7) is a quick check to ensure that all the xhtml is well-formed (tags properly closed and no nesting issues), nothing more.

[Doitsu]

Originally Posted by AlanHK

"... king-01.html.bak' exists in the EPUB but is not declared in the OPF manifest."

-- Now this is strange. Save epub. Close. Reopen.
Epubcheck: "No problems found!".
BUT: open epub as zip, the .bak is still there.

That's actually not a Sigil issue. The EpubCheck plugin will grab all files in the temp folder of the book and assemble them to a new epub before running EpubCheck. That's why you might get error messages about unmanifested backup files, when you edit files outside of Sigil.
When you save and reopen the file, Sigil will ignore all unmanifested files and you'll no longer get EpubCheck error messages, because unmanifested backup files won't be copied to the temp folder of the book.
However, they'll remain in the epub until you save it again with Sigil at which time they'll gone for good.