Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 01-13-2019, 08:22 PM   #1
compurandom
Addict
compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.
 
Posts: 351
Karma: 173048
Join Date: Jun 2015
Device: kobo aura h2o
Wikidata GUI

This plugin imports near arbitrary metadata from Wikidata for books that already have a wikidata ID. This includes a bulk search feature to try to do exact title/author searches in wikidata to find books and can try to match books by other identifiers as well. Use the wikidata metadata plugin to find and add IDs to books that are not found by these methods.

There's a todo.txt in the zip file with the complaints I personally have and possible future features. Discussion of these or others will prioritize their implementation.

Features supported in version 1.3:
  • Bulk search for books in wikidata using exact match on title, author (faster and less likely to hit query limit than the metadata plugin version)
  • Property, Tag, and Identifier editor to help mange wikidata before merging into books
  • Import from books or manually enter arbitrary wikidata properties and entities and mark for filtering
  • Import from books external identifiers in wikidata for editing
  • Merge in series data with support for a second series in a custom column
  • Identifier editor to manage importing and translating of identifiers back and forth to URIs
  • Convert and search by external identifiers already in book metadata
  • Partially merge selected properties when adding new properties.
  • Mark books to indicate merge status
  • Option to hide uninteresting imported properties and tags
  • Import empty placeholder books for books by same author and books in series

Version History:
Spoiler:

Version 1.3 - 24 Feb 2019

Bulk search for books in wikidata by exact match of title, author
- Use this instead of the Wikidata metadata plugin for faster but less complete results.
- Note: don't select too many books at once or it will fail. (The other plugin gets rate limited on bulk import.)
Add features to import other books in series or by same author
* match existing books
* select books and add empty placeholder books for them

User interface for the book import feature is not yet complete -- suggestions welcome!

Version 1.2.1 - 11 Feb 2019
Fix bug with missing mark_* options in defaults
strip spaces for tags and properties

Version 1.2.0 - 28 Jan 2019

Add support for importing series data
second series supported in a custom column

fixed bug involving merging tags

Version 1.0.1 - 22 Jan 2019
Fix a crash during merge caused by occasional unexpected data from wikidata. This is a quick fix to capture the crash so that the offending book can be marked as in error for manual review and continue with the merge.

A more extensive fix is needed to properly handle conversion to target datatypes and possibly do a better job of merging data based on that.

Version 1.0.0 - 20 Jan 2019
New features:
* Search for books in wikidata by identifier
* Merge button on customize tabs
* Merge button on properties/tags tab can merge just selected properties
* Optionally mark books with merge status
* Option to hide uninteresting imported properties and tags
* convert overdrive gutenberg ids to gutenberg ids
* convert urls to ids from id preferences
* option to disable some delete confirmations
* convert overdrive gutenberg ids to gutenberg ids
* convert and delete urls matching imported IDs
* option to disable some delete confirmations
* option to chain convert IDs
Check forum thread for upcoming features and to give feedback on feature priority.

Bugs fixed:
* sorting now preserves selection
* Fixed a typo in new identifier default tag generation
* Saving ID visibility works now
* Messed with column widths
* Updated missing tooltips
* Performance improvements when updating metadata
* "undefined" dates now update properly

Version 0.3.0 - 13 Jan 2019
Initial release of Wikidata GUI plugin

Majority of planned features are implemented
Most features seem to not be buggy anymore


Known bugs:
Spoiler:
  • Import books dialog doesn't (yet) save anything from matched books (unmatched books can be marked to import)
  • Edited author/title on imported books is not used (yet)
  • Bulk query chokes ("HTTP request too large") if you try too many books at once
  • Bulk query chokes on titles containing quotation marks
  • Handles multivalue fields poorly when inserting into a non-string field (luckily this isn't very common)
  • Probably needs more documentation, please ask questions!
  • Display sometimes doesn't update after merge (why? calibre bug?)
Attached Files
File Type: zip Wikidata-gui.zip (79.6 KB, 1077 views)

Last edited by compurandom; 03-15-2019 at 07:32 AM. Reason: 1.2.1 bug fix release
compurandom is offline   Reply With Quote
Old 01-13-2019, 08:23 PM   #2
compurandom
Addict
compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.
 
Posts: 351
Karma: 173048
Join Date: Jun 2015
Device: kobo aura h2o
This post reserved for future info...
compurandom is offline   Reply With Quote
Advert
Old 01-20-2019, 10:15 PM   #3
compurandom
Addict
compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.
 
Posts: 351
Karma: 173048
Join Date: Jun 2015
Device: kobo aura h2o
I'll keep this message up to date with status of unimplemented features.

I've added support for importing series information.
This now supports all the metadata the metadata plugin supports, except this can import *two* series into a book if you create a custom column for it.

The default settings are really intended as examples, but if anyone thinks I should include more, let me know.

The next features I'm considering are:
  • Improve and clean up book import dialog. (Suggestions welcome!)
  • Book import future features: update books missing series info in matched books; import books not listed as in series, but having next/previous links (red links); generate indexes for books with no wikidata series index from next/previous book links
  • Handle books with multiple editions: pull metadata and pubdate from parent editions and multivalue pubdates; search forward and back for gutenberg editions
  • Advanced metadata merging for multivalue entries: replace overwrite column with a function: replace / if empty / min / max /average? / append? / other? (this would fix several bugs)
  • Merge our custom identifier handler with the calibre ID handler in Preferences->Look and Feel->Book Details->Create rules to convert identifiers into links (this is probably mostly cosmetic to the user but would preserve identifiers independent from this plugin)
  • add special handler for URL type properties (wrap in html, wrap in markdown, or insert as identifier uri/url/urn or try to xlate url to id)

If Any of these interest you or you want some other feature, discuss it below and help me prioritize implementation...

Last edited by compurandom; 03-15-2019 at 07:33 AM.
compurandom is offline   Reply With Quote
Old 02-08-2019, 08:01 PM   #4
tlc
Zealot
tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!
 
Posts: 138
Karma: 50288
Join Date: Feb 2009
Device: KK 3G, iPad
My first book attempt:

calibre 3.39.1 embedded-python: True is64bit: True
Darwin-17.7.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '17.7.0', 'Darwin Kernel Version 17.7.0: Fri Nov 2 20:43:16 PDT 2018; root:xnu-4570.71.17~1/RELEASE_X86_64')
Python 2.7.15
OSX: ('10.13.6', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Quality Check (1, 9, 11) && Goodreads (1, 4, 0) && Clean Metadata (0, 0, 6) && Kindle Collections (1, 7, 23) && Wikidata (1, 3, 0) && Wikidata gui (1, 2, 0) && Embed Comic Metadata (1, 4, 0) && Find Duplicates (1, 6, 3)
Traceback (most recent call last):
File "calibre_plugins.wikidata_gui.ui", line 35, in merge_wikidata
File "calibre_plugins.wikidata_gui.ui", line 228, in merge_actual
KeyError: 'mark_updated'
tlc is offline   Reply With Quote
Old 02-08-2019, 08:11 PM   #5
tlc
Zealot
tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!tlc is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!
 
Posts: 138
Karma: 50288
Join Date: Feb 2009
Device: KK 3G, iPad
Works when "Mark update books" is set in customize -> options.
tlc is offline   Reply With Quote
Advert
Old 02-09-2019, 01:29 AM   #6
compurandom
Addict
compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.
 
Posts: 351
Karma: 173048
Join Date: Jun 2015
Device: kobo aura h2o
Thanks, I'll look into it, but probably not until late sunday.
Not sure I've run it with that off. :-> oops.
I'd prefer to release it with the next feature set, but not sure how long that will take, so I might patch it earlier.
compurandom is offline   Reply With Quote
Old 02-09-2019, 08:12 PM   #7
compurandom
Addict
compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.compurandom can program the VCR without an owner's manual.
 
Posts: 351
Karma: 173048
Join Date: Jun 2015
Device: kobo aura h2o
Quote:
Originally Posted by tlc View Post
Works when "Mark update books" is set in customize -> options.
Looking at the code, I bet if you set the option, close the dialog, and then go back and unset the option, it'll still work.

I think I have a fix; the bug is that I forgot to add a few new options to the defaults table, so if you don't set them, they don't have a value and it chokes.

Note: fixed in 1.2.1, just released.

Last edited by compurandom; 02-11-2019 at 01:23 AM.
compurandom is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Metadata Source Plugin] wikidata compurandom Plugins 38 01-14-2019 07:33 AM
[GUI Plugin] Save Virtual Libraries To Column (GUI) chaley Plugins 8 09-24-2017 09:01 PM
GUI Icons Rellwood Development 1 07-09-2017 11:19 AM
GUI Changes luketheobscure Development 40 07-14-2011 04:23 PM
Mobi2Mobi GUI Batch Jad Kindle Formats 8 10-22-2008 07:22 AM


All times are GMT -4. The time now is 10:33 PM.


MobileRead.com is a privately owned, operated and funded community.