Coping with countless images

[theducks]

Originally Posted by Vroni

yes they all have a surrounding <div class="figure">






Unfortunately not. They differ in pixel size like 400*80 or 401*80 or 402*79 and so on. And even if they have the same dimension, the picture itself is not exactly at the same position in the canvas - resulting in slightly different file size.

You don't worry about (that ) Size. as long as you can identify ONLY those fleurons.
OTOH even if the trigger is not exclusive: Search Either Replace-next or Search (a skip) again Have the preview pane visible . It is amazing how fast you can bounce thru 50 uses

[BeckyEbook]

OK. Improved version.
It works in two ways:
a) if the file is identical, it will of course change the name of the other files to the reference name.
b) if the file is not identical but has a width that is in the range given - it will also change its name to the reference name.

If "Width From" or "Width To" is 0, point b) will not be taken into account.

[Tex2002ans]

Originally Posted by Vroni

Unfortunately not. They differ in pixel size like 400*80 or 401*80 or 402*79 and so on. And even if they have the same dimension, the picture itself is not exactly at the same position in the canvas - resulting in slightly different file size.

In this case, I would also run ImageMagick's -trim on it, similar to what I did in this thread (Posts #6-7):

Converting pdf to png images

That would crop out that surrounding whitespace.

(This is the one and only time I unzip an EPUB manually, run conversion on the images, then drag/drop the corrected images back into the EPUB, then reopen in Sigil.)

[JSWolf]

Originally Posted by Tex2002ans

In this case, I would also run ImageMagick's -trim on it, similar to what I did in this thread (Posts #6-7):

Converting pdf to png images

That would crop out that surrounding whitespace.

(This is the one and only time I unzip an EPUB manually, run conversion on the images, then drag/drop the corrected images back into the EPUB, then reopen in Sigil.)

You can crop out the whitespace using Calibre's editor.

[Tex2002ans]

Originally Posted by JSWolf

You can crop out the whitespace using Calibre's editor.

Didn't know this was functionality built in. How do you do it?

[theducks]

Originally Posted by Tex2002ans

Didn't know this was functionality built in. How do you do it?

In the book editor (AZw3 or EPUB), Open the image file. there will be basic image tools on the section toolbar

You can also open the image from the code usage by right clicking.

[Tex2002ans]

Originally Posted by theducks

In the book editor (AZw3 or EPUB), Open the image file. there will be basic image tools on the section toolbar

You can also open the image from the code usage by right clicking.

Thanks. Didn't know that was possible.

But I just tested, and it looks like it's only manual. You have to manually drag a box and tell Calibre to trim.

Having to do that for every fleuron?

Imagemagick's -trim is automated.

Complete Side Note: I was just looking in The Expanse books... and they have a 91x69px fleuron of a little lightning bolt... 80.7 KB JPG... I was able to convert to 1.2 KB PNG... talk about unoptimized! It's ~6% of the total size of the book!

[theducks]

Originally Posted by Tex2002ans

Thanks. Didn't know that was possible.

But I just tested, and it looks like it's only manual. You have to manually drag a box and tell Calibre to trim.

Having to do that for every fleuron?

Imagemagick's -trim is automated.

Complete Side Note: I was just looking in The Expanse books... and they have a 91x69px fleuron of a little lightning bolt... 80.7 KB JPG... I was able to convert to 1.2 KB PNG... talk about unoptimized! It's ~6% of the total size of the book!

It amazes me the amount of inefficient garbage being sold out there.
You can really tell the folk who don't know what they are doing by the variety :^) of styling between chapters

[Vroni]

Originally Posted by theducks

even if the trigger is not exclusive: Search Either Replace-next or Search (a skip) again Have the preview pane visible . It is amazing how fast you can bounce thru 50 uses

Hmmm, ok renaming one of the fleuron to be the master. Then search img(\d+)\.jpg, and cycle through them. First match. No indication available, so replace and watch the preview pane. Oops, that wasnt a fleuron. So click into codeview and hit cmd-z. Click on Find again. Click on Replace. Damn. Again a Non-fleuron. Click into code view (as search window has the focus and CMD-z is not doing an undo if the focus is not helkd by the code view window.

And so on and so on. If you have 200 pictures, and 70 of them are the same fleuron stored multipls times, this is a boring, error-prone manual work. If this happens once ore twice, i wouldnt have thought about another solution. And i'm not the only one having this cases.

And I find it highly interesting how other people deal with Sigil. For example, I would never have thought of deleting the images in the report and sorting them out by renaming them.

[Tex2002ans]

Originally Posted by Vroni

No indication available, so replace and watch the preview pane. Oops, that wasnt a fleuron.

In Code View, you can also Right-Click an image's filename and View Image or Open Tab for Image:



If you "Open Tab", it'll highlight the file in the Book Browser too, so you can easily know which one to rename:



(This is one thing Sigil does WAY better than Calibre's Editor. When you have a huge list of files+images, Sigil keeps the active one visible within Book Browser at all times.)

(And with drag/droppable tabs back in Sigil [yay!], it makes sorting through a mess of files like this much easier. )

Originally Posted by Vroni

So click into codeview and hit cmd-z. Click on Find again. Click on Replace. Damn. Again a Non-fleuron. Click into code view (as search window has the focus and CMD-z is not doing an undo if the focus is not helkd by the code view window.

I agree though that syncing while clicking between Preview/Code View is... very subpar.

It's one of the handful of reasons why I much preferred Book View over Preview.

Originally Posted by Vroni

And I find it highly interesting how other people deal with Sigil. For example, I would never have thought of deleting the images in the report and sorting them out by renaming them.

It's why it's always good to have discussions on these forums, then everyone puts their minds together and figures out the best ways and learns together.

Originally Posted by theducks

It amazes me the amount of inefficient garbage being sold out there.

Looks like it was just a ton of buried EXIF + XMP metadata (probably Photoshop bloat from Print in order to recreate the original high-resolution file). I don't know enough about image metadata to rip it out in a human-readable way.

Originally Posted by theducks

You can really tell the folk who don't know what they are doing by the variety :^) of styling between chapters

Reminds me of a conversion I did years ago. It was hundreds of articles with an "end mark" or "tombstone" at the end ∎:

https://www.fonts.com/content/learni...yphs/end-marks
https://en.wikipedia.org/wiki/Tombstone_(typography)

Seems like it's a print-holdover thing...

Well, in the ebook, the conversion company decided to have microscopic-sized JPGs. Hundreds and hundreds of them... every article had its own "unique" black square.

[KevinH]

Please note that individual graphics are often used to "watermark" some epubs in order to originally identify who the book was sold too. Information can also be encoded right into the pixel values themselves (steganography) that can not be detected visually, and further "watermarks" can be also be found in hex css fields such as colour values.

I have found 2 books that used multiple ways to hide their "watermarking". Never understood why on earth you would hide a watermark but .... could this be one of those rare cases?

[Hitch]

Originally Posted by KevinH

Please note that individual graphics are often used to "watermark" some epubs in order to originally identify who the book was sold too. Information can also be encoded right into the pixel values themselves (steganography) that can not be detected visually, and further "watermarks" can be also be found in hex css fields such as colour values.

I have found 2 books that used multiple ways to hide their "watermarking". Never understood why on earth you would hide a watermark but .... could this be one of those rare cases?

To keep the buyer from removing it. ;-) (Seriously, that's why.)

Hitch

[Tex2002ans]

Originally Posted by KevinH

Please note that individual graphics are often used to "watermark" some epubs in order to originally identify who the book was sold too.

Hmmm... do you happen to recall which sites/stores you noticed this from?

I don't believe any major retailers do such a thing, and just host the same exact file to send to all customers.

All I recall is reading about it every so often, like in this The Digital Reader article:

https://the-digital-reader.com/2016/...watermark-drm/

In addition to adding a unique serial number to the names of files found inside the Epub ebook, Booxtream also embeds the original buyer's name and email on the title page as well as in a footer at the end of each chapter. The digital watermarks can also be found in image metadata and the CSS file, and there's a time stamp which records the specific time the original ebook was downloaded.

Originally Posted by KevinH

I have found 2 books that used multiple ways to hide their "watermarking". Never understood why on earth you would hide a watermark but .... could this be one of those rare cases?

I acquired another copy and compared it to my Kobo version. Nope, exact same image.

If anything, these kinds of things just cause savvy pirates to purchase 2 separate copies, then compare both files together to see exactly what's changed.

Originally Posted by KevinH

Never understood why on earth you would hide a watermark

Used all the time in those huge movie leaks and catching pirates who release screeners (like the movies they send to judges for award shows):

https://torrentfreak.com/replacing-d...irates-190126/
https://torrentfreak.com/oscars-scre...me-low-190226/

Definitely much easier to hide robust watermarks in images/video though... but ebooks?

Things like Calibre EPUB->EPUB or MOBI->EPUB conversions can remove a lot of that cruft.

[AlanHK]

Originally Posted by Vroni

i have an epub with tons of pictures. Round about 50% is just a fleuron, and all of them are the same.

The filenames are all numbered in sequence, so there is no chance to identify a picture by its name.

I do this semi-manually using Irfanview (on Windows).
I've set Irfanview up as an "open with" program for images in Sigil.
So I open any image. Now I press "T" to open the thumbnail viewer, which shows all the images in the temp folder that Sigil has unzipped the images to. Then I can look at the names and images and see if there is a pattern.
If there is a sequence of filenames all the same image, I can select them in Sigil's file list and rename them to "f01" which will name them all in a sequence. Then back to Irfanview's thumbnails, press "F5" and it refreshes to show the new names. Keep doing that till I have all the fleurons renamed. Then I look at the images closely, see if they are identical, if not choose a good one to be the unique one, name it "fleuron".
Back in Sigil, do a regex s&r to convert all the f\d\d\.jpg to fleuron.jpg.

Tools/delete unused media files. Refresh the thumbnail view to see if I got them all.

While we're doing fleurons, I use Irfanview to clean them up. I have Photoshop, but Irfanview is simpler and faster.

Mostly the fleuron comes as a JPEG, often 24 bit, colour.
If it's linework, I rename the file to "Fleuron.gif", then "open with" Irfanview. It complains the format is wrong, I ignore that.
Image/Convert to greyscale.
If it needs it, "Sharpen".
Or Image/Colour corrections and adjust gamma and contrast.
Then "Image/Reduce colour depth". 4 colours is usually enough. If there are greys, 16. Or 2 colours if it's really simple, like a box. Just control-Z if it pixellates and use a higher number.
Now "Image/Palette/Edit palette" and make sure the "black" is pure black. (RGB 0 0 0), and white is (255 255 255).
And you can use F12 to get a very simple painting tools to tidy it up.

Now save (not "save as").
Since you've changed the format, in the OPF change
media-type="image/jpeg"/ to media-type="image/gif"/

-- I use GIF since it is very compact and lossless, and is compatible with Kindle; convert with Kindlegen and it is unchanged. PNGs get converted sometimes to GIF, or sometimes to JPEG, all larger and usually blurrier than the original.

If the fleuron image has a lot of tones, like photo, so better to keep as JPEG. You can use Irfanview's "Options/Lossless crop" to trim it without degrading the image. That will leave a backup image in the temp folder, but Sigil will discard it after you close and reopen the file as it is unmanifested. Or you can refresh Irfan's thumbnails and delete it there.

Unifying the dupes not only reduces the epub filesize by often half, it also make editing the file simpler, if you have 100 fewer files to deal with.

If there are several images I want to edit I will extract them and then reimport, and I can use Irfanview's batch operations to clean up groups of images (e.g., drop caps) en masse.

[exaltedwombat]

Did this reduce the file size by a significant amount?