Removing Image File References from OPF

[Trester99]

For the last 4 weeks I have spent an incredible number of hours working with AI to develop a Calibre Plugin that will identify books with Duplicate Covers, and provide the user an easy means of deleting these duplicate covers. The plugin code has been completed and I am currently in testing/debug mode.

The program really is amazing, there simply is nothing that can duplicate its capabilities. But having said that, as engineer I am really hoping for a better solution than what AI has developed for removing the duplicate cover. From the beginning, AI has insisted that removing the code from the OPF that references the duplicate image is too problematic. The solution that they have implemented is to simply delete the image file itself and leave the reference in the OPF. This method does work, and I am not aware of any negative consequences to this strategy. But I have to admit, I would much rather find a solution that performs a clean removal of the duplicate cover.

This plugin is not for people who have small libraries, it is intended for users who have large, or very large collections. It simply is not an option to use the book editor to clean up that image and the associated reference. I know that for my collection, if I have to choose between duplicate covers, or some stray code, I will choose the removal of duplicate covers.

I do know that converting the book can fix the OPF file, but converting has potential issues also. I've looked at the capabilities of Polish, and the available plugins and I don't see a solution from that side. So, I am hoping that someone can provide suggestions for achieving a clean removal.

[BetterRed]

Did you look at the code for the "Remove missing file entries from manifest" option in the Modify EPUB plugin?

BR

[Trester99]

Originally Posted by BetterRed

Did you look at the code for the "Remove missing file entries from manifest" option in the Modify EPUB plugin?

BR

I did try that but it doesn't remove the line referencing the duplicate image.

[JSWolf]

Kindle eBooks in KF8 do have a duplicate cover. One is the cover and the other is a smaller version which is mot likely the thumbnail. That's intentional. I've seen very few ePub with a duplicate cover.

[BetterRed]

Originally Posted by Trester99

I did try that but it doesn't remove the line referencing the duplicate image.

FTR: I have not looked at the Modify EPUB code.

My thinking was that you could 'copy/paste/adjust" the code from Modify EPUB that implements the Remove missing file entries from manifest option, such that it gets executed after the AI generated code removes the duplicate image.

BR

[Trester99]

Originally Posted by JSWolf

Kindle eBooks in KF8 do have a duplicate cover. One is the cover and the other is a smaller version which is mot likely the thumbnail. That's intentional. I've seen very few ePub with a duplicate cover.

If we are talking about retail epub's, I 100% agree with you! But most duplicates covers are caused by books going thru a conversion. I have no idea what program is causing it, but that program creates a "titlepage.html" file that they include first in the spine, and it creates a "Cover.html" which is second in the spine. Included in the both html files is the "same" image file "cover.jpeg", of course resulting in a duplicate cover. In the program the goal was to get as many duplicate covers into a category that the user could auto-delete. We call that category "Identical" and it represents 80% of the total duplicates. What is really fascinating is that every dup image is visually identical to the cover, though the analytics that define the characteristics of the image can vary somewhat. The only explanation is the conversion process.

[Trester99]

Originally Posted by BetterRed

FTR: I have not looked at the Modify EPUB code.

My thinking was that you could 'copy/paste/adjust" the code from Modify EPUB that implements the Remove missing file entries from manifest option, such that it gets executed after the AI generated code removes the duplicate image.

BR

Your question made me take another look at AI's process. Turns out that they stopped deleting the image file, and are in fact removing the entry from the source html file. I tried using the Modify Epub "Remove unused image files" but that didn't work, I'm assuming because it is still listed in the manifest? But if I had AI go back and delete the image file again that may be the start of the solution. And you are absolutely right that if I could get permission from Kiwidude to use that part of his code, the problem is solved. Hey Kiwi, I've donated twice to your plugins! Thanks BR!

[kovidgoyal]

Not sure what AI you were using but after getting the file name, this is literally 5 lines of code.

from calibre.ebooks.oeb.polish import get_container
from calibre.ebooks.oeb.polish.replace import remove_links_to
c = get_container(path, tweak_mode=True)
def predicate(name, href, fragment=None): return name == fname_to_remove
remove_links_to(c, 
c.remove_item(name_to_remove)
c.commit(path_to_save_to)

[JSWolf]

Originally Posted by Trester99

If we are talking about retail epub's, I 100% agree with you! But most duplicates covers are caused by books going thru a conversion. I have no idea what program is causing it, but that program creates a "titlepage.html" file that they include first in the spine, and it creates a "Cover.html" which is second in the spine. Included in the both html files is the "same" image file "cover.jpeg", of course resulting in a duplicate cover. In the program the goal was to get as many duplicate covers into a category that the user could auto-delete. We call that category "Identical" and it represents 80% of the total duplicates. What is really fascinating is that every dup image is visually identical to the cover, though the analytics that define the characteristics of the image can vary somewhat. The only explanation is the conversion process.

Is the cover image being duplicated or is there two sets of code to display the image?

[jackie_w]

Originally Posted by kovidgoyal

Not sure what AI you were using but after getting the file name, this is literally 5 lines of code.

from calibre.ebooks.oeb.polish import get_container
from calibre.ebooks.oeb.polish.replace import remove_links_to
c = get_container(path, tweak_mode=True)
def predicate(name, href, fragment=None): return name == fname_to_remove
remove_links_to(c, 
c.remove_item(name_to_remove)
c.commit(path_to_save_to)

@Kovid, if you're still reading this thread...

I've been using the calibre get_container function for many years and I'm slightly concerned by this tweak_mode=True parameter. I'm pretty sure I've never used it (at least not knowingly). Yet I have, over the years, written various plugins (User and Editor) which appeared to work OK.

What have I missed? Can you explain, in simple terms, when it should be used and when it's OK to leave it as default tweak_mode=False. Many thanks.

[Trester99]

Originally Posted by kovidgoyal

Not sure what AI you were using but after getting the file name, this is literally 5 lines of code.

from calibre.ebooks.oeb.polish import get_container
from calibre.ebooks.oeb.polish.replace import remove_links_to
c = get_container(path, tweak_mode=True)
def predicate(name, href, fragment=None): return name == fname_to_remove
remove_links_to(c, 
c.remove_item(name_to_remove)
c.commit(path_to_save_to)

Thanks Kovid!

Up until recently, I had been using Gemini Pro and ChatGPT Pro interchangeably. Since Gemini upgraded to 3.0, I have switched to Gemini. Unfortunately, neither Gemini or ChatGPT is ever successful converting a script file to a plugin. For all of my previous projects I have relied on script files ran thru the Calibre Debug program, which works fine. For this program, I tried Grok for the first time and they were able to create a Plugin in about 10 minutes.

[Trester99]

Originally Posted by JSWolf

Is the cover image being duplicated or is there two sets of code to display the image?

It is two set of code in the example above. But in the other duplicates, we do commonly see that "identical" image files are being created. If I remember correctly, AI was using 6 different apps that assessed different characteristics of the images and based on those analytics, you could tell that the 2nd image originated from the cover file.

[kovidgoyal]

@jackie_w: tweak_mode is what the edit book tool uses. Polish Books does not. tweak_mode is generally less forgiving of bad input and does less auto correction of syntx errors andthe like when parsing things. Whether you use it or not depends on whether you want autocorrection or not.

[jackie_w]

Originally Posted by kovidgoyal

@jackie_w: tweak_mode is what the edit book tool uses. Polish Books does not. tweak_mode is generally less forgiving of bad input and does less auto correction of syntx errors andthe like when parsing things. Whether you use it or not depends on whether you want autocorrection or not.

Thanks! That's a relief! I don't use my personal plugins to do bulk updates on any epubs which haven't already passed both calibre CheckBook and EpubCheck, so I think I'm OK.

Merry Christmas