Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 11-07-2014, 04:33 PM   #1
WS64
WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.
 
WS64's Avatar
 
Posts: 641
Karma: 506380
Join Date: Aug 2010
Location: Germany
Device: Kobo Aura / PB Lux 2 / Bookeen Frontlight / Kobo Mini / Nook Color
[GUI Plugin] Clean Metadata

My first plugin...

This plugin edits the metadata of selected books in your library.
You can either just mark the books that will get changed or directly execute the changes.
The main purpose is to mass-update many books at once that have pretty messed up metadata.

Main Features:
  • Cleanup German umlauts in title: "Boser Ueberfall und Duell der Daemonen" => "Boser Überfall und Duell der Dämonen"
  • Cleanup titles: "This is a book title (German Edition)" / "This is a book title. A novel" / "This_is_a_book_title" => "This is a book title"
  • Get series info from titles: "Harry Potter 3: The Prisoner of Azkaban" => title: "The Prisoner of Azkaban", series: "Harry Potter", series_index: 3
  • Cleanup authors: "Lastname, A B.C. Firstname; Anotherlastname, Anotherfirstname" => 1: "A. B. C. Firstname Lastname", 2: "Anotherfirstname Anotherlastname"
  • Swap author and title if title matches an existing author: title: "Stephen King", author: "Cujo" => title: "Cujo", author: "Stephen King" *if* there is already one othebook in your library with author "Stephen King"

Special Notes:
  • Requires Calibre 2.0.0 or later
  • Use at your own risk! Due to some false findings you might create more damage to your metadata than you will repair using this plugin!
  • So far I completely failed to save the settings of the two checkboxes, the interfacedemo plugin was no great help for this. If someone could help me here that would be great, but beware, I'm a complete newbie to Python and Qt5!

Installation Notes:
  1. Download the attached zip file and install the plugin/restart Calibre/add to context menu as described in the Introduction to plugins thread.

Running from command line:
  • not supported

Paypal Donations:
  • nope. I wrote this mainly for myself and don't want to send money to myself!

Version History:

Spoiler:

Version 0.0.6 - 22 Nov 2014
Renamed from "WS64 Cleanup" to "Clean Metadata"

Version 0.0.5 - 20 Nov 2014
New function: Find Duplicate author candidates

Version 0.0.4 - 09 Nov 2014
Bugfixes and minor improvements

Version 0.0.3 - 07 Nov 2014
Initial release. Since just three days ago I have not seen any Python/Qt5 source code the code is far from perfect and probably still very buggy.

Attached Thumbnails
Click image for larger version

Name:	screenshot.jpg
Views:	1490
Size:	44.6 KB
ID:	130745  
Attached Files
File Type: zip Clean Metadata.zip (13.4 KB, 49790 views)

Last edited by WS64; 11-22-2014 at 04:09 AM.
WS64 is offline   Reply With Quote
Advert
Old 11-07-2014, 05:20 PM   #2
jhowell
Wizard
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 1,529
Karma: 12179915
Join Date: Nov 2011
Location: Florida
Device: iPad Air 2, Kindle PW2, Nexus 7, Fire 5th Gen
Quote:
Originally Posted by WS64 View Post
If someone has an idea how I can remove the "libprs500" from the progressbar window please let me know!
As Kovid mentioned in another thread you need to use setWindowTitle. I tried your plugin and saw the libprs title. I modified main.py to add
Code:
        progress.setWindowTitle("WS64 CleanUp")
just after
Code:
        progress = QProgressDialog("Processing books","Stop" , 0,len(ids) ,self)
and it worked for me.
jhowell is offline   Reply With Quote
Old 11-08-2014, 02:35 AM   #3
WS64
WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.
 
WS64's Avatar
 
Posts: 641
Karma: 506380
Join Date: Aug 2010
Location: Germany
Device: Kobo Aura / PB Lux 2 / Bookeen Frontlight / Kobo Mini / Nook Color
@jhowell, thanks, worked!
I'm not sure why it didn't when I tried exactly this the first time...
WS64 is offline   Reply With Quote
Old 11-17-2014, 05:32 PM   #4
DaltonST
Deviser
DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.
 
DaltonST's Avatar
 
Posts: 883
Karma: 71334
Join Date: Aug 2013
Location: Texas
Device: 8" Win10 Tablet w/Calibre64
Umlauts in QuarantineAndScrub: Title Rules Table

Quote:
Originally Posted by WS64 View Post
[*]Cleanup German umlauts in title: "Boser Ueberfall und Duell der Daemonen" => "Boser Überfall und Duell der Dämonen"
@WS64,

If you install the add-on/plug-in QuarantineAndScrub, be sure to add your umlaut word changes to table _title_rules. I would imagine that changing all "ue" and "ae" dipthongs to u's and a's with umlauts would be incorrect, since I noticed that Duell has no umlaut.

The Tag Rules table, _tag_rules, also would need to have entries for Tags that routinely should have an umlaut.


DaltonST
DaltonST is offline   Reply With Quote
Old 11-18-2014, 03:34 AM   #5
WS64
WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.
 
WS64's Avatar
 
Posts: 641
Karma: 506380
Join Date: Aug 2010
Location: Germany
Device: Kobo Aura / PB Lux 2 / Bookeen Frontlight / Kobo Mini / Nook Color
Hi DaltonST, thanks for your feedback.
I guess our plugins have similar goals. ;)
However, the installation of your QuarantineAndScrub looks too complicated for me to try it...

You are right of course that I don't change all "ue"s to "ü"s. I use a blacklist to avoid false changings. Still not perfect, but close...
WS64 is offline   Reply With Quote
Advert
Old 12-02-2014, 10:52 AM   #6
dijks005
Junior Member
dijks005 began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Dec 2014
Device: none
Hi, i am using yr plugin since yesterday. Great plugin!

However using it today i encoutered a problem. The title check is stopping at 61%.

This is the fault report:

calibre, version 2.11.0
FOUT: Niet afgehandelde fout: <b>error</b>:unbalanced parenthesis

calibre 2.11 [64bit] isfrozen: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.8
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: Anobii Fetcher && Fantastic Fiction && Barnes & Noble && Search The Internet && Quality Check && Find Duplicates && Extract ISBN && Goodreads && BOL_NL && Clean Metadata
Traceback (most recent call last):
File "calibre_plugins.CleanMetadata.main", line 125, in <lambda>
File "calibre_plugins.CleanMetadata.main", line 596, in process_books
File "calibre_plugins.CleanMetadata.main", line 379, in process_titles
File "re.py", line 142, in search
File "re.py", line 244, in _compile
error: unbalanced parenthesis

Is there something i can do?

tks
dijks005 is offline   Reply With Quote
Old 12-02-2014, 11:01 AM   #7
WS64
WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.WS64 ought to be getting tired of karma fortunes by now.
 
WS64's Avatar
 
Posts: 641
Karma: 506380
Join Date: Aug 2010
Location: Germany
Device: Kobo Aura / PB Lux 2 / Bookeen Frontlight / Kobo Mini / Nook Color
I stumbled upon the same problem the other day. Seems to happen when the author looks something like this: "abc(def" (no closing brackets).
Havn't checked any further so far (no time right now).
But maybe you can check if you find a book like this in your collection and check if the error goes away if you exclude that one.
Of course this is something I have to fix...
WS64 is offline   Reply With Quote
Old 12-02-2014, 12:23 PM   #8
dijks005
Junior Member
dijks005 began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Dec 2014
Device: none
OK, i will try to find such one. The plugin stopped at about 60%. How is yr plugin processing, starting at record 1, so i can skipp the first 60% of the database?
dijks005 is offline   Reply With Quote
Old 12-02-2014, 05:01 PM   #9
dijks005
Junior Member
dijks005 began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Dec 2014
Device: none
Hi, I checked all 10k+ titles and not such an example is present in my database.
dijks005 is offline   Reply With Quote
Old 03-05-2015, 06:04 AM   #10
a_librarian
Junior Member
a_librarian began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Mar 2015
Device: SONY PRS-650
Hi,
The thread is quite old already, but maybe you're still reading here an have the time to reply.

I just had a look at your great plugin and I found it very useful, especially in terms of correcting German titles.
I also like the feature were you can extract the series from the title, but it doesn't work too well for me, probably because it looks for a specific format in the titles ('series' 'series_index': 'title', right?). When the format is not right, for example because it's 'author' - 'series' - 'series_index' - 'title', or just 'series' 'series_index', it does't find any entries.
Is there a way to give the plugin a certain string instead of the default one?
For example: I have a list of books, were the author or title field contain the 'series' 'series_index'. I would like to just bulk-copy that to the series field, instead of having to do it manually.

Thanks very much for the good work!
a librarian
a_librarian is offline   Reply With Quote
Old 03-05-2015, 12:12 PM   #11
DaltonST
Deviser
DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.DaltonST knows who John Galt is.
 
DaltonST's Avatar
 
Posts: 883
Karma: 71334
Join Date: Aug 2013
Location: Texas
Device: 8" Win10 Tablet w/Calibre64
https://www.mobileread.com/forums/sho...40048#poststop
DaltonST is offline   Reply With Quote
Old 03-06-2015, 02:56 AM   #12
a_librarian
Junior Member
a_librarian began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Mar 2015
Device: SONY PRS-650
Thanks very much for the link - that looks very promising! I don't know how I missed this before...
a_librarian is offline   Reply With Quote
Old 02-09-2017, 08:51 AM   #13
AnselmD
Zealot
AnselmD began at the beginning.
 
Posts: 105
Karma: 10
Join Date: Oct 2013
Device: none
"Clean Metadata" Window is corrupt

Hi,

I am using Ubuntu 16.04 LTS.

"CleanupGerman Umlauts in Title:" -> "Mark Only": the task is done, but the "Clean Metadata" Window is still there and has a corrupt content. I is not possible to close it. Even in in calibre Debug modus I do not see an error.

Attached Thumbnails
Click image for larger version

Name:	Screenshot from 2017-02-09 14-35-26.png
Views:	1274
Size:	61.6 KB
ID:	154865  
AnselmD is offline   Reply With Quote
Old 04-16-2017, 03:29 PM   #14
jasonreadmobile
Junior Member
jasonreadmobile began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Apr 2017
Device: kobo
Quote:
Originally Posted by dijks005 View Post
Hi, I checked all 10k+ titles and not such an example is present in my database.
A bit of a necro-thread... I ran into the same problem as others with processing stopping midway. The plugin code needs to be modified to treat the fields it reads from the metadata more carefully.
The use of re.search with raw strings in the python code is interpreting "regex-like" strings from the title and author fields as regexes. Specific cases of "[2003-LABEL]" or something similar are seen as invalid character class ranges.
I modified my local version of the code to preprocess the title and author strings with "re.escape(str)" prior to use to protect against this.
I don't know how to resubmit my changes for the plugin, the owner can probably fix this if he/she cares.
jasonreadmobile is offline   Reply With Quote
Old 04-21-2017, 07:06 AM   #15
AnselmD
Zealot
AnselmD began at the beginning.
 
Posts: 105
Karma: 10
Join Date: Oct 2013
Device: none
Quote:
Originally Posted by jasonreadmobile View Post
I don't know how to resubmit my changes for the plugin, the owner can probably fix this if he/she cares.
Last Activity of Ian_Stott: 07-29-2012 09:23 PM

You can try to send a personal mail.

For another plugin, i add a version to the thread, see here:

https://www.mobileread.com/forums/sh...9&postcount=24
AnselmD is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Zotero Metadata Importer DaltonST Plugins 126 08-04-2017 10:26 AM
[GUI Plugin] Consolidate All Library Metadata (CALM) DaltonST Plugins 188 06-09-2017 11:20 AM
Call fetch/download metadata from GUI plugin BeckyEbook Development 0 02-03-2017 06:51 PM
Getting custom metadata field in GUI plugin SauliusP. Development 7 02-11-2012 01:31 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 07:24 PM.


MobileRead.com is a privately owned, operated and funded community.