Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 06-22-2016, 11:13 AM   #1
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
[GUI Plugin] Zotero Metadata Importer

[GUI Plugin] Zotero Metadata Importer

Summary:

You must use Zotero to use ZMI, which both imports files from Zotero into Calibre, and exports files from Calibre for import into Zotero.

  • ZMI has the ability to export Calibre metadata to an RIS file specifically formatted for Zotero, including an "L1" RIS Tag used by Zotero to automatically import a Calibre .pdf file into Zotero as an Item's File Attachment, and an "L4" RIS Tag that does the same for other ebook formats, such as .epub or .mobi or any others.
  • If you want your Zotero book files in Calibre along with their Zotero metadata, ZMI is currently the only known mechanism to do so other than manually keying it into Calibre. Refer to the attached images that provide an overview.

    You must use ZMI to add your Zotero books to Calibre if you want ZMI to automatically update those books with the Zotero metadata. Do not use the "Add Books" button in Calibre unless you want to manually enter the Zotero Metadata.
  • ZMI has a "Compare to Zotero" function that will use the Zotero "KEY", which is assigned automatically by Zotero to each and every item, and which is imported by ZMI into Calibre, to compare Zotero items to Calibre books. This comparison is by both Collection in Zotero, and for all Calibre books in your ZMI Calibre Library regardless of Collection assignment.

    This comparison will automatically update Calibre metadata for each book with its currently assigned Zotero Collection. If a Zotero item is part of more than one (1) Collection, ZMI will report that fact.

    See the example ZMI Comparison Report for all Collections below.

    Spoiler:



    Full Listing of All Zotero Collections and Items Compared to Calibre


    ------------------------------------------


    Grand Total Collection Items in Zotero: 48
    Grand Total Zotero Items in Calibre: 2


    ------------------------------------------


    Standard 37VM3FBC Digital sound recorder: a case study on designing embedded systems using the UML notation id: 13867


    Standard 49QMRUPD A critical customer identification model for technology and service providers id: 13866


    Standard 4F4MT6AR *MISSING: Vorgehensweise bei einer qualitativen Expertenbefragung am Beispiel Fondsmanager journalArticle


    Standard 6Y57NUZB *MISSING: A scientist's view.pdf attachment application/pdf


    Standard 72P258HM *MISSING: ETHER AND THE THEORY OF RELATIVITY bookSection


    Standard 9YA8AJJR *MISSING: 3722-1.pdf attachment application/pdf


    Standard AETQ35R2 *MISSING: Asymmetrische Verschlüsselung (mathematische Grundlagen) journalArticle


    Standard AEU7AJHZ *MISSING: Adaptive MIMO-OFDM for future mobile radio communications conferencePaper


    Standard BVMM3W74 *MISSING: The Post-Embargo Open Access Citation Advantage: It Exists (Probably), It’s Modest (Usually), and the Rich Get Richer (of Course) journalArticle


    Standard C3MAYEKW *MISSING: Persistence of Different-sized Populations: A inEmpirical Assessment of Rapid Extinctions in Bighorn Sheep journalArticle


    Standard CD5ATU78 *MISSING: An Introduction to Ontologies and Ontology Engineering bookSection


    Standard CMVPBQBN *MISSING: Effect of Sociality and Season on Gray Wolf (Canis lupus) Foraging Behavior: Implications for Estimating Summer Kill Rate journalArticle


    Standard D5VLTK4W *MISSING: Front-loading problem solving in co-development: managing the contractual, organisational and cognitive dimensions journalArticle


    Standard DQ35VHCQ *MISSING: THE POTENTIAL OF THREE-DIMENSIONAL DISPLAY- TECHNOLOGIES FOR THE VISUALIZATION OF GEO- VIRTUAL ENVIRONMENTS journalArticle


    Standard DUYS9WV4 *MISSING: Open access, readership, citations: a randomized controlled trial of scientific journal publishing journalArticle


    Standard E8DKSHFB *MISSING: [THESIS TITLE GOES HERE] - Georgia Tech Thesis Office.pdf attachment application/pdf


    Standard ESLXLLTL *MISSING: Quantifying Landscape Ruggedness for Animal Habitat Analysis: A Case Study Using Bighorn Sheep in the Mojave Desert journalArticle


    Standard FQ8QHGI4 *MISSING: A lightweight methodology for rapid ontology engineering journalArticle


    Standard FU36AL86 *MISSING: Neonatal mortality of elk driven by climate, predator phenology and predator community composition journalArticle


    Standard GA9QN984 *MISSING: The Challenges and Issues with Nanotechnology at the Product Development Stage journalArticle


    Standard GBCHIRDB *MISSING: 2.pdf attachment application/pdf


    Standard HYAAVK9M *MISSING: Ontology Engineering in the Era of Linked Data | ASIS&T journalArticle


    Standard I8YQ4YJD *MISSING: Open Access Citation Advantage: An Annotated Bibliography journalArticle


    Standard IDN2HM4M *MISSING: Molecular Surveillance for Bartonella, Borrelia, and Rickettsia Species in Ticks from Desert Bighorn Sheep (Ovis canadensis) and Mule Deer (Odocoileus hemionus) in Southern California, USA journalArticle


    Standard J4NSBRB9 *MISSING: Highways block gene flow and cause a rapid decline in genetic diversity of desert bighorn sheep journalArticle


    Standard L53K2XQT *MISSING: Yellowstone after Wolves journalArticle


    Standard M94CL4EK *MISSING: 8Nanomat_Szene_Lambauer_Voss_IER.pdf attachment application/pdf


    Standard NDY7NA2F *MISSING: Corynebacterium pseudotuberculosis and Copper Deficiency in a Male Rocky Mountain Bighorn Sheep (Ovis canadensis canadensis) in Utah, USA journalArticle


    Standard NZ69YTNT *MISSING: Von der Hochschule ins Berufsleben journalArticle


    Standard PB2KLSCV *MISSING: nanoDE-Report_2011.pdf attachment application/pdf


    Standard QMQ9DPX2 *MISSING: The future of entrepreneurial ®nance journalArticle


    Standard QNM7SFWI *MISSING: 10_EN_303.pdf attachment application/pdf


    Standard SDQ9XWIC *MISSING: 1461.de.pdf attachment application/pdf


    Standard T5A3C4XN *MISSING: H.R.5751 - 114th Congress (2015-2016): Shiras Moose Protection and Recovery Act bill


    Standard TJNAFX8X *MISSING: big_data_the_management_revolution.pdf attachment application/pdf


    Standard UJW9JGPB *MISSING: PROF. DR. RUDOLF VOLKART journalArticle


    Standard UPK5MVT4 *MISSING: DOES ONLINE AVAILABILITY INCREASE CITATIONS? THEORY AND EVIDENCE FROM A PANEL OF ECONOMICS AND BUSINESS JOURNALS journalArticle


    Standard USYZVXUG *MISSING: big_data_the_management_revolution.pdf attachment application/pdf


    Standard VKJYN8G4 *MISSING: Predator-prey size relationships in an African large-mammal food web journalArticle


    Standard W5WXTCC8 *MISSING: Predicting the evolutionary consequences of trophy hunting on a quantitative trait journalArticle


    Standard W77ACTU9 *MISSING: KlausurLoesungUeT1-1_4.4.2003.dvi - Unknown.pdf attachment application/pdf


    Standard X95SHFEQ *MISSING: Direct and indirect effects of early-life environment on lifetime fitness of bighorn ewes journalArticle


    Standard XZ49MKK7 *MISSING: Doing what your neighbour does: neighbour proximity, familiarity and postural alignment increase behavioural mimicry journalArticle


    Standard Y4C4S5BR *MISSING: Thoughts on Cloning Humans.pdf attachment application/pdf


    Standard YLQ5UGTG *MISSING: Wolf kill rates across winter in a low-density moose system in Alaska journalArticle



    Collection Items - Total: 45


    Collection Items - Missing: 43


    Collection Items - Match: 2


    ------------------------------------------


    Standard Duplicate1 37VM3FBC Digital sound recorder: a case study on designing embedded systems using the UML notation id: 13867


    Standard Duplicate1 49QMRUPD A critical customer identification model for technology and service providers id: 13866



    Collection Items - Total: 2


    Collection Items - Missing: 0


    Collection Items - Match: 2


    ------------------------------------------


    Standard Duplicate2 37VM3FBC Digital sound recorder: a case study on designing embedded systems using the UML notation id: 13867



    Collection Items - Total: 1


    Collection Items - Missing: 0


    Collection Items - Match: 1


    ------------------------------------------
    Total Number of Zotero Collections: 3
    ------------------------------------------


    All Items - Total: 48


    All Items - Missing: 43


    All Items - Match: 5


    ------------------------------------------


    Grand Total Collection Items in Zotero: 48
    Grand Total Zotero Items in Calibre: 2


    ------------------------------------------
    No Orphan Calibre 'ZKeys' found. All Calibre 'ZKeys' are also in Zotero.
    ------------------------------------------


  • The ZMI "User Guide" is comprised of all of the ToolTips throughout the ZMI Tabs.


What is Zotero?
Spoiler:

Zotero is free and open-source reference management software to manage bibliographic data and related research materials (such as PDF files). Notable features include web browser integration, online syncing, generation of in-text citations, footnotes and bibliographies, as well as integration with the word processors Microsoft Word, LibreOffice, OpenOffice.org Writer and NeoOffice. It is produced by the Center for History and New Media at George Mason University (GMU).

For more information: Zotero Overview



Workaround for File Attachment Issues:
Spoiler:

Scenario: ZMI will import your Zotero CSV file perfectly, but it cannot find your actual Zotero File Attachments in your Zotero storage directory. The reason is unknown, but you want to start using Calibre for your Zotero File Attachments and Metadata as soon as possible. This is how:

[1] Set your ZMI options to create empty books if no file attachments are found.

[2] Run ZMI in auto-step mode with your 'real' CSV file.

[3] Finish the updating of the new empty books' metadata in ZMI by clicking the remaining pushbuttons etc.

[4] Navigate to your Zotero 'storage' folder.

[5] Drag-and-drop each .pdf onto the Calibre Book Detail pane (on the right) for the appropriately selected "empty book". That will be easy, since the Zotero storage files have the Zotero Key in their path, and Calibre has it in the ZKey Custom Column. You can also match on author and title, of course. Calibre will ask you if you really want to add that new format. Say "yes".

You will then have your Zotero .pdf files in Calibre with all of the appropriate ZMI-created Calibre metadata.

When the reason for your temporary problem is solved, you can cease using this Drag-and-drop workaround.




Requires Minimum Calibre Version: 6.0.0 Desktop Version Only

Requires Minimum Zotero Version: 6.0.15 Desktop Version Only

Old & Deprecated versions of ZMI: https://www.mobileread.com/forums/sh...d.php?t=327472

Other Useful Calibre Plugins to Consider:


Version History:
Spoiler:

Version 1.0.78 - 2023-01-12 Qt.core.
Version 1.0.77 - 2022-11-30 Miscellany. Minimum Calibre version now 6.0.0.
Version 1.0.76 - 2022-04-14 Miscellany. Minimum Calibre version now 5.99.9.
Version 1.0.75 - 2022-04-06 Enhanced the Comparison Tab Feature "Show Full Listing [All Books]"
Version 1.0.74 - 2022-04-05 Collection Comparison ToolTips enhanced. New Comparison Tab Feature: Show Full Listing [All Books].
Version 1.0.73 - 2022-04-03 Collection assignment now uses a 3d potential #zotero_key Custom Column value in addition to the Calibre Identifiers zkey and zkey_file. This 3d source is rarely useful, but compensates for "drift" between Zotero and Calibre for items originally imported from a Zotero exported .csv file.
Version 1.0.72 - 2022-03-25 Qt6 Compatibility for Calibre Version 5.99.8+. Technical changes.
Version 1.0.70 - 2020-12-14 Technical changes.
Version 1.0.69 - 2020-12-13 Compatibility changes for Calibre 5.7. Minimum Version now 5.7.
Version 1.0.68 - 2020-07-20 Qt tweaks so the ZMI dialog displays the same in Linux on a 14" Chromebook with Debian as it does in Windows 10 on a 21" display.
Version 1.0.67 - 2020-07-13 RIS File export: add new pushbutton for exporting L1/L4 Tags that have the absolute path of the respective Calibre book (instead of the path of the intermediate temporary file) to provide the ability to "Link" to the original Calibre book rather than importing into Zotero the physical book file itself. Caution: "Linking" a Calibre book instead of "Storing" it in Zotero disables the ZMI "Collection Comparison" function for each Linked book, since by definition the Linked book does not physically exist in Zotero storage.
Version 1.0.66 - 2020-07-09 Python 3 compatibility, especially for users who had never previously first used Python 2 ZMI.
Version 1.0.65 - 2020-04-24 Regression testing and a general tune-up. Suggestion: Add the 'Mark Books' action to the 'Context Menu' via 'Preferences > Toolbars & Menus > Context Menus'.
Version 1.0.64 - 2020-04-02 RIS tag "AU": {authors} OR {author_sort}; ToolTips for RIS Export configuration revised accordingly.
Version 1.0.63 - 2020-01-03 Technical changes after Python 3.8 testing with Calibre 4.99.2
Version 1.0.62 - 2020-01-02 Technical changes after Python 3.8 testing with Calibre 4.99.2
Version 1.0.61 - 2019-11-15 Miscellaneous Technical Changes.
Version 1.0.60 - 2019-05-30 Python 3 compatibility. Minimum Calibre version now 3.41.3.
Version 1.0.59 - 2018-04-12 Miscellaneous enhancements.
Version 1.0.58 - 2018-04-11 New: Add the Zotero Key to each just-exported Calibre book after its import into Zotero so that the ZMI:Comparison Tab functionality may be used for both books imported from Zotero and Zotero books imported from Calibre.
Version 1.0.57 - 2018-04-07 New: RIS Export Tag L4 for Ebook file attachments
Version 1.0.56 - 2018-04-07 New: RIS Export Tag L1 for PDF file attachments
Version 1.0.55 - 2018-04-05 Beta: Activated RIS Export Tag L1 for PDF file attachments
Version 1.0.54 - 2018-04-04 Beta: Added RIS Export Tag L1 for .PDF Files
Version 1.0.53 - 2017-10-22 New ZMI option to create an empty book if the sole file attachment is html.
Version 1.0.52 - 2017-10-21 Technical tweaks re: showing all incomplete books.
Version 1.0.51 - 2017-09-23 Miscellaneous tweaks.
Version 1.0.50 - 2017-09-22 Calibre 3.8 now required. Update Calibre prior to updating ZMI. Functionality changes for Calibre 3.8.0 compatibility related to setting the 'covers' for newly imported Zotero 'books'.
Version 1.0.49 - 2017-08-04 New ZMI Tab: Export RIS File. Export Calibre metadata as .ris file designed specifically for import into Zotero.
Version 1.0.48 - 2017-07-28 Fixed regression error in mark/select new ZMI book interaction.
Version 1.0.47 - 2017-07-27 Miscellaneous tweaks.
Version 1.0.46 - 2017-07-26 Miscellaneous Enhancements.
Version 1.0.45 - 2017-07-25 Fixed item "title".
Version 1.0.44 - 2017-07-25 Comparison Tab enhancements.
Version 1.0.43 - 2017-07-24 Enhancement to capture Zotero Collection for all Calibre books regardless of whether it is a file attachment or not.
Version 1.0.42 - 2017-07-23 Technical changes for Zotero 5.0 Compatibility (e.g. change in column name in Zotero table itemAttachments).
Version 1.0.41 - 2017-06-11 Technical changes for Calibre 3.0.0 compatibility.
Version 1.0.40 - 2017-05-21 Technical changes.
Version 1.0.39 - 2017-05-12 Technical changes for Calibre 3.0.0 compatibility.
Version 1.0.38 - 2017-05-05 Allow themes with user-defined icons.
Version 1.0.37 - 2017-01-17 ToolTip and other miscellaneous tweaks.
Version 1.0.36 - 2017-01-14 Technical changes to account for variability in Zotero .csv file data.
Version 1.0.35 - 2016-08-12 "List Failed Adds" fixed.
Version 1.0.34 - 2016-08-12 Implemented the new features of Calibre 2.64 Custom Columns datatype of 'comments'. Zotero Custom Columns have been changed to always be 'long-text' with the heading on the 'side' of the book detail pane.
Version 1.0.33 - 2016-07-20 Changed temporary file creation for "empty books" to avoid failures.
Version 1.0.32 - 2016-07-20 Enhancement to the handling of complex .html files having many associated images and javascript files that need to be zipped together prior to adding to Calibre.
Version 1.0.31 - 2016-07-19 Repair any invalid or corrupt CSV 'file attachment' path values during the import process.
Version 1.0.30 - 2016-07-19 New: User options to copy Zotero automatic and/or manual tags to Calibre Tags.
Version 1.0.29 - 2016-07-18 Changes to 'Compare Zotero Database to Calibre Database' to take advantage of the new Identifier 'zkey_file'.
Version 1.0.28 - 2016-07-18 Major enhancements. New Identifier: zkey_file. New user option to import .txt, .text, .html and .htm file attachments as Calibre "books", although Calibre will automatically convert the html files into .zip formatted books. Those must then be converted from .zip to .epub to be readable in an e-reader. New user option to import Zotero items that have no file attachments.
Version 1.0.27 - 2016-07-15 Miscellaneous enhancements; New Calibre Identifier of 'zcollection' is updated for books with their matching Zotero collection name automatically whenever the Compare function is executed for their Zotero collection.
Version 1.0.26 - 2016-07-15 New: Compare Zotero Database to Calibre Database to find Zotero books missing from Calibre.

Version 1.0.25 - 2016-07-13 Miscellaneous enhancements.
Version 1.0.24 - 2016-07-12 Miscellaneous enhancements. Requires Minimum Calibre Version 2.62 to be installed. Upgrade Calibre first.
Version 1.0.23 - 2016-07-07 New: 'Auto-Select Multiple CSV Files' then Auto-Process all of them simultaneously. Afterwards, archive or delete the original CSV files.
Version 1.0.22 - 2016-07-06 Miscellaneous enhancements.
Version 1.0.21 - 2016-07-05 Technical enhancements.
Version 1.0.20 - 2016-07-04 Technical enhancements.
Version 1.0.19 - 2016-07-03 Enhancements to increase the efficiency of the ZMI 'process'.
Version 1.0.18 - 2016-07-02 Miscellany.
Version 1.0.17 - 2016-07-01 Many small enhancements to increase the efficiency of the ZMI 'process'.
Version 1.0.16 - 2016-06-30 New options to enhance the efficiency of the ZMI 'process'.
Version 1.0.15 - 2016-06-30 New: Progress Dialog with ability to cancel; miscellaneous tweaks; ToolTip enhancements.
Version 1.0.14 - 2016-06-30 Major Enhancements.
Version 1.0.13 - 2016-06-28 Changes.
Version 1.0.12 - 2016-06-28 Major Enhancements.
Version 1.0.11 - 2016-06-28 Enhancements.
Version 1.0.10 - 2016-06-27 Major enhancements.
Version 1.0.9 - 2016-06-27 New CSV-related error messages to assist in correcting CSV data issues
Version 1.0.8 - 2016-06-27 Enhancements.
Version 1.0.7 - 2016-06-26 Enhancements.
Version 1.0.6 - 2016-06-26 Major enhancements.
Version 1.0.5 - 2016-06-25 Major enhancements.
Version 1.0.4 - 2016-06-24 New Identifier: DOI (Document Object ID) added from ZDoi. New messages while calibredb is adding missing books.
Version 1.0.3 - 2016-06-22 New: Detects Keys for Zotero books that do not exist anywhere in Calibre in the ZKey (#zotero_key) Custom Column and also not anywhere in the Identifiers table, and offers to run calibredb to add them. Calibredb then adds the book while also adding an Identifier of zkey with a value of the CSV key to the Calibre book. Automated matching of the Identifier zkey with the CSV to automatically update metadata from Zotero is then possible.
Version 1.0.2 - 2016-06-22 Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts.
Version 1.0.1 - 2016-06-22 Initial release.
Attached Thumbnails
Click image for larger version

Name:	zmi_custom_columns_version_1_0_0.jpg
Views:	4602
Size:	202.2 KB
ID:	149586   Click image for larger version

Name:	zmi_example_v1027.jpg
Views:	5601
Size:	1.20 MB
ID:	150244   Click image for larger version

Name:	zmi_all_tabs_version_1_0_34.jpg
Views:	3218
Size:	411.0 KB
ID:	150876   Click image for larger version

Name:	zmi_zcollection_identifier.jpg
Views:	2732
Size:	1.00 MB
ID:	158085   Click image for larger version

Name:	zmi_export_ris_custom_configured_for_zotero.jpg
Views:	2353
Size:	906.5 KB
ID:	158253   Click image for larger version

Name:	zmi_tag_L1_example.jpg
Views:	2360
Size:	1.02 MB
ID:	163294   Click image for larger version

Name:	zmi_ris_all.jpg
Views:	1931
Size:	804.4 KB
ID:	163300   Click image for larger version

Name:	zmi_tag_L4.jpg
Views:	1867
Size:	917.5 KB
ID:	163318   Click image for larger version

Name:	zmi_retrieve_zkey_from_new_zotero_attachments.jpg
Views:	2131
Size:	1.04 MB
ID:	163427   Click image for larger version

Name:	collection comparison - show full listing example.jpg
Views:	492
Size:	201.8 KB
ID:	193128   Click image for larger version

Name:	zotero_storage_sqlite_example_attachment.jpg
Views:	306
Size:	1.07 MB
ID:	198231   Click image for larger version

Name:	DaltonSTplugins@20230122.jpg
Views:	255
Size:	420.6 KB
ID:	199198  
Attached Files
File Type: zip zotero_metadata_importer.zip (68.8 KB, 21679 views)

Last edited by DaltonST; 02-06-2023 at 12:03 PM. Reason: Version 1.0.78
DaltonST is offline   Reply With Quote
Old 06-22-2016, 11:13 AM   #2
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
Reserved for future use.
DaltonST is offline   Reply With Quote
Advert
Old 06-22-2016, 11:53 AM   #3
excaliber
Connoisseur
excaliber began at the beginning.
 
excaliber's Avatar
 
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
Thanks for the plugin!

I just tried it and I am getting following errors after pressing the first button:

calibre, version 2.59.0
ERROR: ZMI: GENERATION OF CUSTOM COLUMNS TERMINATED PREMATURELY........ERROR: >>>>'module' object has no attribute 'STARTUPINFO'

calibre, version 2.59.0
ERROR: Automatically Add Custom Columns: Fatal error experienced in calibredb. If you use the Apple OSX operating system, then it is probable that you have not already installed the Calibre OSX 'Command-Line' Utilities, which include calibredb. Unless you install them, or if you do not use OSX, you will have to add the Custom Columns manually.
excaliber is offline   Reply With Quote
Old 06-22-2016, 12:43 PM   #4
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
@excaliber:

I use Windows 10, and it works perfectly every time.

What OS do you have?

Did you read the Original Post section about Special OS Notes?


DaltonST
DaltonST is offline   Reply With Quote
Old 06-22-2016, 12:55 PM   #5
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
@excaliber:

I bet you use Linux. Based on quick research, it appears Linux cannot handle STARTUPINFO like Windows can.

I will upload a theoretical fix, but I do not have Linux so I cannot test it. You can be the guinea pig.

Thanks.

DaltonST
DaltonST is offline   Reply With Quote
Advert
Old 06-22-2016, 01:11 PM   #6
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
New version posted with a subprocess module change for OSs that are not Windows. Windows still works perfectly, but I cannot test Linux or OSX.

DaltonST


n.b. Newly uploaded versions are not visible in Calibre's 'Plugin Updater' until after the bottom of the hour plus a few minutes or so. ZMI is new, so that will be problematic for a while.

To get it immediately, you must manually download it from the OP and import it from that .zip file via Preferences > Plugins > Import from File

Last edited by DaltonST; 06-22-2016 at 01:43 PM.
DaltonST is offline   Reply With Quote
Old 06-22-2016, 05:29 PM   #7
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts

Version 1.0.2 - 2016-06-22 Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts.

If you do not see it in your Calibre 'Plugin Updater', you will need to download it manually.

If anyone has Linux or OSX, I would like to know if the automatic generation of Zotero Custom Columns works for you viz a viz spawning calibredb in subprocesses to create the new Custom Columns.

Windows 10 works perfectly, of course, because I developed it on Windows 10.


DaltonST
DaltonST is offline   Reply With Quote
Old 06-22-2016, 07:20 PM   #8
excaliber
Connoisseur
excaliber began at the beginning.
 
excaliber's Avatar
 
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
@DaltonST:
I bet you use Linux. Based on quick research, it appears Linux cannot handle STARTUPINFO like Windows can.
I will upload a theoretical fix, but I do not have Linux so I cannot test it. You can be the guinea pig.

Right. I use mostly Linux, I have mentioned this before. I am now testing now your new version.
I get no more the aforementioned errors!
Automatic generation of Zotero Custom Columns works! (I don't understand what do you mean by "spawning calibredb in subprocesses")
I can update title and author of a selected book (ISBN and ISSN haven't tried)
I can manually update calibre metadata for a single book.
I don't get what is really happening at step 2 and 3.
How can I update several books automatically?
The books must be first imported into calibre?
I thought calibre would import all books and their metadata after reading the csv file.

Besides that I would like to make two suggestions:
- Apart from the title and author there could be even more entries which coincide, so the user could be offered the option to associate them and overwrite them too - Maybe in a table?
- An option to make the new Zotero entries appear in the details view of Calibre would be nice.
excaliber is offline   Reply With Quote
Old 06-22-2016, 08:27 PM   #9
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
Answers: Part 1 of 2

Quote:
Originally Posted by excaliber View Post
@DaltonST:
[1] I don't get what is really happening at step 2 and 3.
[2] How can I update several books automatically?
[3] The books must be first imported into calibre?
[4] I thought calibre would import all books and their metadata after reading the csv file.
[1] Have you read the ToolTips by hovering your mouse over each object in the ZMI window? They explain everything. Also, have you analyzed the zmi_overview.jpg attachment to the Original Post? It also explains everything simultaneously.

[2] You can update 10,000+ books simultaneously and automatically. You just have to select a CSV file that has all 10,000+ Zotero book metadata, and select all of their corresponding 10,000+ Calibre books. If you select "all Calibre books", only those with a corresponding entry in the Zotero CSV file will (or can) can be updated. If you read the various ToolTips, you will see that changing the Calibre Author or Calibre Title prior to finalizing the automatic Zotero metadata update process has negative consequences because of the profound changes in the Calibre ebook paths.

[3] Yes. Only Calibre imports books into Calibre.

[4] No. Never will happen. Use Calibre to import books into Calibre, regardless of their origin.

The ToolTips for each button and grey area in the ZMI dialog box explain how ZMI works.



DaltonST

Last edited by DaltonST; 06-22-2016 at 08:34 PM.
DaltonST is offline   Reply With Quote
Old 06-22-2016, 08:32 PM   #10
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
Answers: Part 2 of 2

Quote:
[1] --Apart from the title and author there could be even more entries which coincide, so the user could be offered the option to associate them and overwrite them too - Maybe in a table?

[2] - An option to make the new Zotero entries appear in the details view of Calibre would be nice.
[2] What appears in the Calibre details view is a basic Calibre function, not ZMI. Simply go to Calibre > Preferences > Look & Feel > Book Details.


[1] What other entries should go from the CSV to where in Calibre? Give me a list. That requires specific Python programming.


DaltonST
DaltonST is offline   Reply With Quote
Old 06-23-2016, 09:06 AM   #11
excaliber
Connoisseur
excaliber began at the beginning.
 
excaliber's Avatar
 
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
[3] Yes. Only Calibre imports books into Calibre.

[4] No. Never will happen. Use Calibre to import books into Calibre, regardless of their origin.


Is not possible for Calibre to import files if their path is known? I mean, If there is a file attached in Zotero then this is reflected in the "File Attachments" column.


[2] What appears in the Calibre details view is a basic Calibre function, not ZMI. Simply go to Calibre > Preferences > Look & Feel > Book Details.

One more thing learned. Did that but any ideas why I'm getting 2 columns and why in the 2nd column only the values are showing up (screenshot attached)?

[1] What other entries should go from the CSV to where in Calibre? Give me a list. That requires specific Python programming.
If it is possible, I would suggest to only hard code the fields that are equal in both Zotero and Calibre (default fields) like Title and Authors and give the user the option to do the matching for other fields. For example I have installed the ISBN extraction plugin and Words count plugin or I had already in Calibre a custom column "Entry Type" which corresponds now to the ZItemType. I have attached a file to give you an idea of what I mean.

[1] Have you read the ToolTips by hovering your mouse over each object in the ZMI window? They explain everything. Also, have you analyzed the zmi_overview.jpg attachment to the Original Post? It also explains everything simultaneously.

[2] You can update 10,000+ books simultaneously and automatically. You just have to select a CSV file that has all 10,000+ Zotero book metadata, and select all of their corresponding 10,000+ Calibre books. If you select "all Calibre books", only those with a corresponding entry in the Zotero CSV file will (or can) can be updated. If you read the various ToolTips, you will see that changing the Calibre Author or Calibre Title prior to finalizing the automatic Zotero metadata update process has negative consequences because of the profound changes in the Calibre ebook paths.


Yes, I have read it all but still does't work. I don't know what I am doing wrong.
I am proceeding like this:
I have a folder with books and I import them into Calibre as well as into Zotero.
  1. I fetch the metadata in Zotero.
  2. I export the metadata to a csv file.
  3. I import the csv file into Calibre via ZMI. (1st button)
  4. I highlight the new books in Calibre.
  5. I select the csv file (2nd button)
  6. I try to update the metadata of the new books in Calibre. (3rd button)
-> 0 books updated
The same result when I import the books not from a random folder but from Zotero's storage folder.
So what am I doing wrong? How does the matching between the Zotero entries and the Calibre entries work? With the help of which criterion? How does it find the right entries in Calibre?
Attached Thumbnails
Click image for larger version

Name:	ZMI_Import.png
Views:	1253
Size:	50.1 KB
ID:	149614   Click image for larger version

Name:	calibre_book_details.png
Views:	1139
Size:	44.1 KB
ID:	149615  

Last edited by excaliber; 06-23-2016 at 09:14 AM.
excaliber is offline   Reply With Quote
Old 06-23-2016, 10:12 AM   #12
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
@excaliber:


Quote:
Is not possible for Calibre to import files if their path is known? I mean, If there is a file attached in Zotero then this is reflected in the "File Attachments" column
Yes, which is why I am currently doing final testing on the next release that will detect missing books and ask if you want to use Calibre to add them. I will likely post the new release tonight. See the attached image.

The criterion is Zotero's Key in the CSV being matched against Calibre's Custom Column #zotero_key a.k.a. ZKey.

If you mess with the Calibre ZKey column value for a book, it will no longer match the CSV...not good.

The next release will add missing books by looking for Zotero Keys in the CSV that are not represented anywhere in the current Calibre Library. It will then invoked Calibre's calibredb command-line utility to add the files represented in the CSV field "File Attachments". See note #1 below.


Quote:
For example I have installed the ISBN extraction plugin and Words count plugin
By definition those 2 plug-ins are hardcoded to do only a single thing (ISBN and Word Count/Pages) by reading the .pdf text, and parsing it.


Quote:
How does the matching between the Zotero entries and the Calibre entries work? With the help of which criterion? How does it find the right entries in Calibre?
It matches by finding a Calibre book Path with a book filename that is identical to the Zotero file attachment filename. It ignores everything in the Paths except the final filename. If you change the Calibre filename in any way, it will no longer match. Hence, the strong warning about not changing Calibre Author or Title prior to completing the ZMI update.

Perhaps you should create a new Calibre Library, and start from scratch importing the books from Zotero, and then running ZMI before touching the books at all.

You can then move the books from your old Library to the new, 'uncorrupted' Library. Just be sure to set your Calibre > Preferences for merging duplicates properly first.

Note #1: for the auto-add missing Keys to work, all of your Zotero books must be in the same Calibre Library for the same logical Zotero "library" equivalent. Otherwise, books would be deemed as missing even though you had previously added them and then moved them to another Library. Not good.



DaltonST
Attached Thumbnails
Click image for larger version

Name:	zmi_new_add_missing_books_question_dialog.jpg
Views:	1062
Size:	60.8 KB
ID:	149616  

Last edited by DaltonST; 06-23-2016 at 10:50 AM. Reason: Changes
DaltonST is offline   Reply With Quote
Old 06-23-2016, 12:04 PM   #13
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
@excaliber:

The use of the upcoming auto-add via calibredb will result in a 100% match between the CSV and Calibre, since I fully control what calibredb does. It will add the Zotero Key as a Calibre Identifier for each book as it is added. The automatic metadata update will match on that if it exists.

The Identifier zkey:xxxxxx will not exist for legacy books already in your Calibre Library.

For the latter, the automatic matching still has to match on filenames being exactly the same.

Unfortunately, Calibre truncates filenames when they get too long, which means that those books will have to be manually matched to the CSV file.

There are also differences between Windows and Linux as regards Calibre filename restrictions.

Truncation of the filename for non-auto-added books results in a 100% failure rate for automatic matching for updating the metadata from the CSV.

Truncation is irrelevant when books are added via the upcoming auto-add via calibredb, since the books will be matched by the new Identifer zkey:xxxxxx which will be added by calibredb automatically and immediately.

You might seriously consider "starting fresh" with a new Calibre Library for Zotero, and populating it exclusively using the upcoming auto-add process



DaltonST
DaltonST is offline   Reply With Quote
Old 06-23-2016, 03:48 PM   #14
DaltonST
Deviser
DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.DaltonST ought to be getting tired of karma fortunes by now.
 
DaltonST's Avatar
 
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
Version 1.0.3 Posted

Version 1.0.3 - 2016-06-22 New: Detects Keys for Zotero books that do not exist anywhere in Calibre in the ZKey (#zotero_key) Custom Column and also not anywhere in the Identifiers table, and offers to run calibredb to auto-add them. Calibredb then adds the book while also adding an Identifier of zkey with a value of the CSV key to the Calibre book. Automated matching of the Identifier zkey with the CSV to automatically update metadata from Zotero is then possible.

See the attached explanatory image.

An important note: The Zotero CSV file field "file attachments" must correspond exactly to the path of your Zotero books (file attachments). If you get virtually a zero % success rate in automatically adding the Zotero books to Calibre, you should strongly suspect that the data in the CSV field "file attachments" does not exactly equal the path of your Zotero books in your OS. Check for non-displayable characters in the CSV that of course do not exist in the paths of the books, making finding the books via the CSV file attachments field impossible.




DaltonST
Attached Thumbnails
Click image for larger version

Name:	zmi_auto_add_missing_books_example.jpg
Views:	1283
Size:	880.8 KB
ID:	149623  
DaltonST is offline   Reply With Quote
Old 06-23-2016, 06:47 PM   #15
excaliber
Connoisseur
excaliber began at the beginning.
 
excaliber's Avatar
 
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
@DaltonST:
It matches by finding a Calibre book Path with a book filename that is identical to the Zotero file attachment filename. It ignores everything in the Paths except the final filename. If you change the Calibre filename in any way, it will no longer match. Hence, the strong warning about not changing Calibre Author or Title prior to completing the ZMI update.

Calibre is renaming the files when I import them. That's why it didn't work. After your hint I tried this:
Imported some files into Calibre.
I then imported the associated files the Library directory of Calibre (they were already renamed) into Zotero.
Then I I followed the procedure I mentioned before and it worked.
Then I did something I didn't know it was possible - I did drag'n'drop some files from Calibre to Zotero. This has also worked and and it's much easier...
It doesn't bother me if it's working like this, I only found weird that I have first to import files into Calibre and then into Zotero and then back to Calibre. Is this the normal procedure?


By definition those 2 plug-ins are hardcoded to do only a single thing (ISBN and Word Count/Pages) by reading the .pdf text, and parsing it.


OK. But why are you mentioning this? Did you see the attached screenshots? I mentioned these plugins as an example - a user can have already other fields and columns which could theoretically then assigned to the Zotero fields. In the case of these plugins: ISBN and Pages. I had already other custom columns in my Calibre configuration before I start using Zotero which could then be assigned to Zotero fields. If there is no option to unite them then they can appear twice in the Calibre columns. To me it doesn't make sense but I'm happy with this version. Again, it's only a suggestion...

The fact that I'm getting 2 columns in the details view is because of Calibre or ZMI? Or why in the 2nd column only the values are showing up?

Last edited by excaliber; 06-23-2016 at 07:10 PM.
excaliber is offline   Reply With Quote
Reply

Tags
ris, zotero

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Consolidate All Library Metadata (CALM) DaltonST Plugins 333 12-15-2023 08:40 PM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 492 10-25-2022 08:13 AM
[GUI Plugin] OPML Importer (RSS feeds from feedly/google reader) fem Plugins 14 07-19-2014 10:41 AM
Getting custom metadata field in GUI plugin SauliusP. Development 7 02-11-2012 01:31 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 10:49 PM.


MobileRead.com is a privately owned, operated and funded community.