[idea] Remove duplicates

un_pogaz · 05-14-2019, 08:50 AM

Some books contain duplicate images (sometimes +10 duplicate)
With a little trickery, the Regex and the unused media deletion tool, you can delete a lot of duplicates.
But in some cases, there may be a different image in the middle, or even several duplicate images.

This is where the idea for this plugin comes from:
1) The plugin opens a wall of thumbnail displaying the images of the book (or a check-list with a preview area for quick consultation)
2) We select the images that we consider identical
3) The plugin "merges" the images (renames them with a Regex like this one below, then deletes the images that have become unused)

(Images/0002.jpg|Images/0003.jpg|Images/0006.jpg|Images/0008.jpg)
>
Images/0002.jpg

Thank you in advance.

AlanHK · 05-25-2019, 01:03 AM

Yeah, it's really annoying to open a file and see it has 50 names for some ornament, all indistinguishable, but not binary identical.

I use Irfanview's Thumbnail view, then rename in Sigil to a common prefix, then use a regex to make all similar images link to one image, remove unused images.

This can reduce the filesize by 50% or more and make the code more intelligible.

theducks · 05-25-2019, 12:35 PM

I use Report: Images part of the editor.
You can sort by size and visually see the image.
Then I REGEX the books Image code (NEVER use REPLACE ALL) AND Replace 1-at-a-time that I am really sure are identical (subtle diffs has got me before).
Then I rescan Reports, and select n delete those with 0 uses (yes, you can delete from reports)

DNSB · 05-25-2019, 03:25 PM

Quote:

Originally Posted by theducks

Then I rescan Reports, and select n delete those with 0 uses (yes, you can delete from reports)

That's a handy bit of knowledge.

un_pogaz · 05-28-2019, 03:21 AM

Quote:

Originally Posted by theducks

I use Report: Images part of the editor.

What/Where is it's this feature?

BetterRed · 05-28-2019, 03:30 AM

Quote:

Originally Posted by un_pogaz

What/Where is it's this feature?

Tools->Reports->Image Files

BR

05-14-2019, 08:50 AM	#1
un_pogaz Chalut o/ Posts: 410 Karma: 145324 Join Date: Dec 2017 Device: Kobo	[idea] Remove duplicates images Some books contain duplicate images (sometimes +10 duplicate) With a little trickery, the Regex and the unused media deletion tool, you can delete a lot of duplicates. But in some cases, there may be a different image in the middle, or even several duplicate images. This is where the idea for this plugin comes from: 1) The plugin opens a wall of thumbnail displaying the images of the book (or a check-list with a preview area for quick consultation) 2) We select the images that we consider identical 3) The plugin "merges" the images (renames them with a Regex like this one below, then deletes the images that have become unused) (Images/0002.jpg\|Images/0003.jpg\|Images/0006.jpg\|Images/0008.jpg) > Images/0002.jpg Thank you in advance. Last edited by un_pogaz; 05-16-2019 at 08:50 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
easiest way to remove duplicates	rjherald	Library Management	2	07-02-2016 02:48 PM
Remove duplicates + images	absconditus	Recipes	0	09-24-2014 07:20 PM
Duplicates handling with Find Duplicates plugin	erfjr	Calibre	0	03-05-2013 02:52 PM
Found duplicates - but how to remove?	Motomaggot	Calibre	7	02-07-2012 09:22 PM
My "read" tag idea enhancement for Calibre idea	rcuadro	Calibre	10	01-20-2011 04:23 PM

05-25-2019, 01:03 AM	#2
AlanHK Guru Posts: 668 Karma: 929286 Join Date: Apr 2014 Device: PW-3, iPad, Android phone	Yeah, it's really annoying to open a file and see it has 50 names for some ornament, all indistinguishable, but not binary identical. I use Irfanview's Thumbnail view, then rename in Sigil to a common prefix, then use a regex to make all similar images link to one image, remove unused images. This can reduce the filesize by 50% or more and make the code more intelligible.

05-25-2019, 12:35 PM	#3
theducks Well trained by Cats Posts: 29,803 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I use Report: Images part of the editor. You can sort by size and visually see the image. Then I REGEX the books Image code (NEVER use REPLACE ALL) AND Replace 1-at-a-time that I am really sure are identical (subtle diffs has got me before). Then I rescan Reports, and select n delete those with 0 uses (yes, you can delete from reports)

Advert

Advert