Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 06-22-2016, 11:13 AM   #1
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
[GUI Plugin] Zotero Metadata Importer

[GUI Plugin] Zotero Metadata Importer

Summary:

You must use Zotero to use ZMI. If you want your Zotero book files in Calibre along with their Zotero metadata, ZMI is currently the only known mechanism to do so other than manually keying it into Calibre. Refer to the attached images that provide an overview.

You must use ZMI to add your Zotero books to Calibre if you want ZMI to automatically update those books with the Zotero metadata.

Do not use the "Add Books" button in Calibre unless you want to manually enter the Zotero Metadata.

The ZMI "User Guide" is represented by all of the ToolTips throughout the ZMI Tabs.


What is Zotero?
Spoiler:

Zotero is free and open-source reference management software to manage bibliographic data and related research materials (such as PDF files). Notable features include web browser integration, online syncing, generation of in-text citations, footnotes and bibliographies, as well as integration with the word processors Microsoft Word, LibreOffice, OpenOffice.org Writer and NeoOffice. It is produced by the Center for History and New Media at George Mason University (GMU).

For more information: Zotero Overview


Special Operating System Notes:
Spoiler:


[1] Automatic creation of new Custom Columns and adding Zotero books to Calibre uses Calibre's 'calibredb' command-line utility. Windows automatically installs calibredb, but OSX does not. OSX users must install the Calibre command-line utilities separately. Search www.mobileread.com for instructions.

[2] Windows 7 users who are not 'administrators' and have not turned off 'UAC' will have fatal errors when using ZMI. The UAC issue will be related to ZMI spawning an operating system process to run calibredb. You will have to always execute Calibre from a shortcut that has been configured to always run as an 'administrator' if you wish to use ZMI.

[3] If you find yourself needing the .json configuration file for ZMI, then that is a symptom that may be resolved by implementing [4] and/or [5], below.

[4] If you use Calibre Portable: CALIBRE_TEMP_DIR is used by Calibre for its temporary file directory, and TMPDIR et al are used by generic Python for its temporary file directory. CALIBRE_TEMP_DIR must be consistent with TMPDIR et al. Implement [5], below. Also, ensure that Calibre is in your PATH.

[5] If you have personalized your "temporary" directories, such as using a RAMDisk for your temporary files, you must set certain standard environment variables such that Python finds and uses the one that you want Python to find and use. From the Python documentation:
Spoiler:



Quote:
The default directory is chosen from a platform-dependent list, but the user of the application can control the directory location by setting the TMPDIR, TEMP or TMP environment variables

Python searches a standard list of directories to find one which the calling user can create files in. The list is:

  • The directory named by the TMPDIR environment variable.
  • The directory named by the TEMP environment variable.
  • The directory named by the TMP environment variable.
  • A platform-specific location:
    On Windows, the directories C:\TEMP, C:\TMP, \TEMP, and \TMP, in that order.
  • On all other platforms, the directories /tmp, /var/tmp, and /usr/tmp, in that order.
  • As a last resort, the current working directory.




Workaround for File Attachment Issues:
Spoiler:

Scenario: ZMI will import your Zotero CSV file perfectly, but it cannot find your actual Zotero File Attachments in your Zotero storage directory. The reason is unknown, but you want to start using Calibre for your Zotero File Attachments and Metadata as soon as possible. This is how:

[1] Set your ZMI options to create empty books if no file attachments are found.

[2] Run ZMI in auto-step mode with your 'real' CSV file.

[3] Finish the updating of the new empty books' metadata in ZMI by clicking the remaining pushbuttons etc.

[4] Navigate to your Zotero 'storage' folder.

[5] Drag-and-drop each .pdf onto the Calibre Book Detail pane (on the right) for the appropriately selected "empty book". That will be easy, since the Zotero storage files have the Zotero Key in their path, and Calibre has it in the ZKey Custom Column. You can also match on author and title, of course. Calibre will ask you if you really want to add that new format. Say "yes".

You will then have your Zotero .pdf files in Calibre with all of the appropriate ZMI-created Calibre metadata.

When the reason for your temporary problem is solved, you can cease using this Drag-and-drop workaround.





Requires Minimum Calibre Version: 2.64
Version 2.64 implements plain text comments Custom Columns that are used by ZMI.

Version History:
Spoiler:
Version 1.0.35 - 2016-08-12 "List Failed Adds" fixed.
Version 1.0.34 - 2016-08-12 Implemented the new features of Calibre 2.64 Custom Columns datatype of 'comments'. Zotero Custom Columns have been changed to always be 'long-text' with the heading on the 'side' of the book detail pane.

Version History Archive:
Spoiler:

Version 1.0.33 - 2016-07-20 Changed temporary file creation for "empty books" to avoid failures.
Version 1.0.32 - 2016-07-20 Enhancement to the handling of complex .html files having many associated images and javascript files that need to be zipped together prior to adding to Calibre.
Version 1.0.31 - 2016-07-19 Repair any invalid or corrupt CSV 'file attachment' path values during the import process.
Version 1.0.30 - 2016-07-19 New: User options to copy Zotero automatic and/or manual tags to Calibre Tags.
Version 1.0.29 - 2016-07-18 Changes to 'Compare Zotero Database to Calibre Database' to take advantage of the new Identifier 'zkey_file'.
Version 1.0.28 - 2016-07-18 Major enhancements. New Identifier: zkey_file. New user option to import .txt, .text, .html and .htm file attachments as Calibre "books", although Calibre will automatically convert the html files into .zip formatted books. Those must then be converted from .zip to .epub to be readable in an e-reader. New user option to import Zotero items that have no file attachments.
Version 1.0.27 - 2016-07-15 Miscellaneous enhancements; New Calibre Identifier of 'zcollection' is updated for books with their matching Zotero collection name automatically whenever the Compare function is executed for their Zotero collection.
Version 1.0.26 - 2016-07-15 New: Compare Zotero Database to Calibre Database to find Zotero books missing from Calibre.

Version 1.0.25 - 2016-07-13 Miscellaneous enhancements.
Version 1.0.24 - 2016-07-12 Miscellaneous enhancements. Requires Minimum Calibre Version 2.62 to be installed. Upgrade Calibre first.
Version 1.0.23 - 2016-07-07 New: 'Auto-Select Multiple CSV Files' then Auto-Process all of them simultaneously. Afterwards, archive or delete the original CSV files.
Version 1.0.22 - 2016-07-06 Miscellaneous enhancements.
Version 1.0.21 - 2016-07-05 Technical enhancements.
Version 1.0.20 - 2016-07-04 Technical enhancements.
Version 1.0.19 - 2016-07-03 Enhancements to increase the efficiency of the ZMI 'process'.
Version 1.0.18 - 2016-07-02 Miscellany.
Version 1.0.17 - 2016-07-01 Many small enhancements to increase the efficiency of the ZMI 'process'.
Version 1.0.16 - 2016-06-30 New options to enhance the efficiency of the ZMI 'process'.
Version 1.0.15 - 2016-06-30 New: Progress Dialog with ability to cancel; miscellaneous tweaks; ToolTip enhancements.
Version 1.0.14 - 2016-06-30 Major Enhancements.
Version 1.0.13 - 2016-06-28 Changes.
Version 1.0.12 - 2016-06-28 Major Enhancements.
Version 1.0.11 - 2016-06-28 Enhancements.
Version 1.0.10 - 2016-06-27 Major enhancements.
Version 1.0.9 - 2016-06-27 New CSV-related error messages to assist in correcting CSV data issues
Version 1.0.8 - 2016-06-27 Enhancements.
Version 1.0.7 - 2016-06-26 Enhancements.
Version 1.0.6 - 2016-06-26 Major enhancements.
Version 1.0.5 - 2016-06-25 Major enhancements.
Version 1.0.4 - 2016-06-24 New Identifier: DOI (Document Object ID) added from ZDoi. New messages while calibredb is adding missing books.
Version 1.0.3 - 2016-06-22 New: Detects Keys for Zotero books that do not exist anywhere in Calibre in the ZKey (#zotero_key) Custom Column and also not anywhere in the Identifiers table, and offers to run calibredb to add them. Calibredb then adds the book while also adding an Identifier of zkey with a value of the CSV key to the Calibre book. Automated matching of the Identifier zkey with the CSV to automatically update metadata from Zotero is then possible.
Version 1.0.2 - 2016-06-22 Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts.
Version 1.0.1 - 2016-06-22 Initial release.
Attached Thumbnails
Click image for larger version

Name:	zmi_custom_columns_version_1_0_0.jpg
Views:	161
Size:	202.2 KB
ID:	149586   Click image for larger version

Name:	zmi_example_v1027.jpg
Views:	55
Size:	1.20 MB
ID:	150244   Click image for larger version

Name:	zmi_all_tabs_version_1_0_34.jpg
Views:	14
Size:	411.0 KB
ID:	150876  
Attached Files
File Type: zip zotero_metadata_importer.zip (47.4 KB, 652 views)

Last edited by DaltonST; 08-15-2016 at 11:27 AM. Reason: Version 1.0.35 Info
DaltonST is offline   Reply With Quote
Old 06-22-2016, 11:13 AM   #2
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
Reserved for future use.
DaltonST is offline   Reply With Quote
Old 06-22-2016, 11:53 AM   #3
excaliber
Enthusiast
excaliber began at the beginning.
 
Posts: 33
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
Thanks for the plugin!

I just tried it and I am getting following errors after pressing the first button:

calibre, version 2.59.0
ERROR: ZMI: GENERATION OF CUSTOM COLUMNS TERMINATED PREMATURELY........ERROR: >>>>'module' object has no attribute 'STARTUPINFO'

calibre, version 2.59.0
ERROR: Automatically Add Custom Columns: Fatal error experienced in calibredb. If you use the Apple OSX operating system, then it is probable that you have not already installed the Calibre OSX 'Command-Line' Utilities, which include calibredb. Unless you install them, or if you do not use OSX, you will have to add the Custom Columns manually.
excaliber is offline   Reply With Quote
Old 06-22-2016, 12:43 PM   #4
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
@excaliber:

I use Windows 10, and it works perfectly every time.

What OS do you have?

Did you read the Original Post section about Special OS Notes?


DaltonST
DaltonST is offline   Reply With Quote
Old 06-22-2016, 12:55 PM   #5
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
@excaliber:

I bet you use Linux. Based on quick research, it appears Linux cannot handle STARTUPINFO like Windows can.

I will upload a theoretical fix, but I do not have Linux so I cannot test it. You can be the guinea pig.

Thanks.

DaltonST
DaltonST is offline   Reply With Quote
Old 06-22-2016, 01:11 PM   #6
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
New version posted with a subprocess module change for OSs that are not Windows. Windows still works perfectly, but I cannot test Linux or OSX.

DaltonST


n.b. Newly uploaded versions are not visible in Calibre's 'Plugin Updater' until after the bottom of the hour plus a few minutes or so. ZMI is new, so that will be problematic for a while.

To get it immediately, you must manually download it from the OP and import it from that .zip file via Preferences > Plugins > Import from File

Last edited by DaltonST; 06-22-2016 at 01:43 PM.
DaltonST is offline   Reply With Quote
Old 06-22-2016, 05:29 PM   #7
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts

Version 1.0.2 - 2016-06-22 Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts.

If you do not see it in your Calibre 'Plugin Updater', you will need to download it manually.

If anyone has Linux or OSX, I would like to know if the automatic generation of Zotero Custom Columns works for you viz a viz spawning calibredb in subprocesses to create the new Custom Columns.

Windows 10 works perfectly, of course, because I developed it on Windows 10.


DaltonST
DaltonST is offline   Reply With Quote
Old 06-22-2016, 07:20 PM   #8
excaliber
Enthusiast
excaliber began at the beginning.
 
Posts: 33
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
@DaltonST:
I bet you use Linux. Based on quick research, it appears Linux cannot handle STARTUPINFO like Windows can.
I will upload a theoretical fix, but I do not have Linux so I cannot test it. You can be the guinea pig.

Right. I use mostly Linux, I have mentioned this before. I am now testing now your new version.
I get no more the aforementioned errors!
Automatic generation of Zotero Custom Columns works! (I don't understand what do you mean by "spawning calibredb in subprocesses")
I can update title and author of a selected book (ISBN and ISSN haven't tried)
I can manually update calibre metadata for a single book.
I don't get what is really happening at step 2 and 3.
How can I update several books automatically?
The books must be first imported into calibre?
I thought calibre would import all books and their metadata after reading the csv file.

Besides that I would like to make two suggestions:
- Apart from the title and author there could be even more entries which coincide, so the user could be offered the option to associate them and overwrite them too - Maybe in a table?
- An option to make the new Zotero entries appear in the details view of Calibre would be nice.
excaliber is offline   Reply With Quote
Old 06-22-2016, 08:27 PM   #9
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
Answers: Part 1 of 2

Quote:
Originally Posted by excaliber View Post
@DaltonST:
[1] I don't get what is really happening at step 2 and 3.
[2] How can I update several books automatically?
[3] The books must be first imported into calibre?
[4] I thought calibre would import all books and their metadata after reading the csv file.
[1] Have you read the ToolTips by hovering your mouse over each object in the ZMI window? They explain everything. Also, have you analyzed the zmi_overview.jpg attachment to the Original Post? It also explains everything simultaneously.

[2] You can update 10,000+ books simultaneously and automatically. You just have to select a CSV file that has all 10,000+ Zotero book metadata, and select all of their corresponding 10,000+ Calibre books. If you select "all Calibre books", only those with a corresponding entry in the Zotero CSV file will (or can) can be updated. If you read the various ToolTips, you will see that changing the Calibre Author or Calibre Title prior to finalizing the automatic Zotero metadata update process has negative consequences because of the profound changes in the Calibre ebook paths.

[3] Yes. Only Calibre imports books into Calibre.

[4] No. Never will happen. Use Calibre to import books into Calibre, regardless of their origin.

The ToolTips for each button and grey area in the ZMI dialog box explain how ZMI works.



DaltonST

Last edited by DaltonST; 06-22-2016 at 08:34 PM.
DaltonST is offline   Reply With Quote
Old 06-22-2016, 08:32 PM   #10
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
Answers: Part 2 of 2

Quote:
[1] --Apart from the title and author there could be even more entries which coincide, so the user could be offered the option to associate them and overwrite them too - Maybe in a table?

[2] - An option to make the new Zotero entries appear in the details view of Calibre would be nice.
[2] What appears in the Calibre details view is a basic Calibre function, not ZMI. Simply go to Calibre > Preferences > Look & Feel > Book Details.


[1] What other entries should go from the CSV to where in Calibre? Give me a list. That requires specific Python programming.


DaltonST
DaltonST is offline   Reply With Quote
Old 06-23-2016, 09:06 AM   #11
excaliber
Enthusiast
excaliber began at the beginning.
 
Posts: 33
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
[3] Yes. Only Calibre imports books into Calibre.

[4] No. Never will happen. Use Calibre to import books into Calibre, regardless of their origin.


Is not possible for Calibre to import files if their path is known? I mean, If there is a file attached in Zotero then this is reflected in the "File Attachments" column.


[2] What appears in the Calibre details view is a basic Calibre function, not ZMI. Simply go to Calibre > Preferences > Look & Feel > Book Details.

One more thing learned. Did that but any ideas why I'm getting 2 columns and why in the 2nd column only the values are showing up (screenshot attached)?

[1] What other entries should go from the CSV to where in Calibre? Give me a list. That requires specific Python programming.
If it is possible, I would suggest to only hard code the fields that are equal in both Zotero and Calibre (default fields) like Title and Authors and give the user the option to do the matching for other fields. For example I have installed the ISBN extraction plugin and Words count plugin or I had already in Calibre a custom column "Entry Type" which corresponds now to the ZItemType. I have attached a file to give you an idea of what I mean.

[1] Have you read the ToolTips by hovering your mouse over each object in the ZMI window? They explain everything. Also, have you analyzed the zmi_overview.jpg attachment to the Original Post? It also explains everything simultaneously.

[2] You can update 10,000+ books simultaneously and automatically. You just have to select a CSV file that has all 10,000+ Zotero book metadata, and select all of their corresponding 10,000+ Calibre books. If you select "all Calibre books", only those with a corresponding entry in the Zotero CSV file will (or can) can be updated. If you read the various ToolTips, you will see that changing the Calibre Author or Calibre Title prior to finalizing the automatic Zotero metadata update process has negative consequences because of the profound changes in the Calibre ebook paths.


Yes, I have read it all but still does't work. I don't know what I am doing wrong.
I am proceeding like this:
I have a folder with books and I import them into Calibre as well as into Zotero.
  1. I fetch the metadata in Zotero.
  2. I export the metadata to a csv file.
  3. I import the csv file into Calibre via ZMI. (1st button)
  4. I highlight the new books in Calibre.
  5. I select the csv file (2nd button)
  6. I try to update the metadata of the new books in Calibre. (3rd button)
-> 0 books updated
The same result when I import the books not from a random folder but from Zotero's storage folder.
So what am I doing wrong? How does the matching between the Zotero entries and the Calibre entries work? With the help of which criterion? How does it find the right entries in Calibre?
Attached Thumbnails
Click image for larger version

Name:	ZMI_Import.png
Views:	70
Size:	50.1 KB
ID:	149614   Click image for larger version

Name:	calibre_book_details.png
Views:	65
Size:	44.1 KB
ID:	149615  

Last edited by excaliber; 06-23-2016 at 09:14 AM.
excaliber is offline   Reply With Quote
Old 06-23-2016, 10:12 AM   #12
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
@excaliber:


Quote:
Is not possible for Calibre to import files if their path is known? I mean, If there is a file attached in Zotero then this is reflected in the "File Attachments" column
Yes, which is why I am currently doing final testing on the next release that will detect missing books and ask if you want to use Calibre to add them. I will likely post the new release tonight. See the attached image.

The criterion is Zotero's Key in the CSV being matched against Calibre's Custom Column #zotero_key a.k.a. ZKey.

If you mess with the Calibre ZKey column value for a book, it will no longer match the CSV...not good.

The next release will add missing books by looking for Zotero Keys in the CSV that are not represented anywhere in the current Calibre Library. It will then invoked Calibre's calibredb command-line utility to add the files represented in the CSV field "File Attachments". See note #1 below.


Quote:
For example I have installed the ISBN extraction plugin and Words count plugin
By definition those 2 plug-ins are hardcoded to do only a single thing (ISBN and Word Count/Pages) by reading the .pdf text, and parsing it.


Quote:
How does the matching between the Zotero entries and the Calibre entries work? With the help of which criterion? How does it find the right entries in Calibre?
It matches by finding a Calibre book Path with a book filename that is identical to the Zotero file attachment filename. It ignores everything in the Paths except the final filename. If you change the Calibre filename in any way, it will no longer match. Hence, the strong warning about not changing Calibre Author or Title prior to completing the ZMI update.

Perhaps you should create a new Calibre Library, and start from scratch importing the books from Zotero, and then running ZMI before touching the books at all.

You can then move the books from your old Library to the new, 'uncorrupted' Library. Just be sure to set your Calibre > Preferences for merging duplicates properly first.

Note #1: for the auto-add missing Keys to work, all of your Zotero books must be in the same Calibre Library for the same logical Zotero "library" equivalent. Otherwise, books would be deemed as missing even though you had previously added them and then moved them to another Library. Not good.



DaltonST
Attached Thumbnails
Click image for larger version

Name:	zmi_new_add_missing_books_question_dialog.jpg
Views:	69
Size:	60.8 KB
ID:	149616  

Last edited by DaltonST; 06-23-2016 at 10:50 AM. Reason: Changes
DaltonST is offline   Reply With Quote
Old 06-23-2016, 12:04 PM   #13
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
@excaliber:

The use of the upcoming auto-add via calibredb will result in a 100% match between the CSV and Calibre, since I fully control what calibredb does. It will add the Zotero Key as a Calibre Identifier for each book as it is added. The automatic metadata update will match on that if it exists.

The Identifier zkey:xxxxxx will not exist for legacy books already in your Calibre Library.

For the latter, the automatic matching still has to match on filenames being exactly the same.

Unfortunately, Calibre truncates filenames when they get too long, which means that those books will have to be manually matched to the CSV file.

There are also differences between Windows and Linux as regards Calibre filename restrictions.

Truncation of the filename for non-auto-added books results in a 100% failure rate for automatic matching for updating the metadata from the CSV.

Truncation is irrelevant when books are added via the upcoming auto-add via calibredb, since the books will be matched by the new Identifer zkey:xxxxxx which will be added by calibredb automatically and immediately.

You might seriously consider "starting fresh" with a new Calibre Library for Zotero, and populating it exclusively using the upcoming auto-add process



DaltonST
DaltonST is offline   Reply With Quote
Old 06-23-2016, 03:48 PM   #14
DaltonST
Deviser
DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'DaltonST knows the difference between 'who' and 'whom'
 
DaltonST's Avatar
 
Posts: 749
Karma: 10260
Join Date: Aug 2013
Location: Texas
Device: 8" Windows 10 Tablet
Version 1.0.3 Posted

Version 1.0.3 - 2016-06-22 New: Detects Keys for Zotero books that do not exist anywhere in Calibre in the ZKey (#zotero_key) Custom Column and also not anywhere in the Identifiers table, and offers to run calibredb to auto-add them. Calibredb then adds the book while also adding an Identifier of zkey with a value of the CSV key to the Calibre book. Automated matching of the Identifier zkey with the CSV to automatically update metadata from Zotero is then possible.

See the attached explanatory image.

An important note: The Zotero CSV file field "file attachments" must correspond exactly to the path of your Zotero books (file attachments). If you get virtually a zero % success rate in automatically adding the Zotero books to Calibre, you should strongly suspect that the data in the CSV field "file attachments" does not exactly equal the path of your Zotero books in your OS. Check for non-displayable characters in the CSV that of course do not exist in the paths of the books, making finding the books via the CSV file attachments field impossible.




DaltonST
Attached Thumbnails
Click image for larger version

Name:	zmi_auto_add_missing_books_example.jpg
Views:	64
Size:	880.8 KB
ID:	149623  
DaltonST is offline   Reply With Quote
Old 06-23-2016, 06:47 PM   #15
excaliber
Enthusiast
excaliber began at the beginning.
 
Posts: 33
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
@DaltonST:
It matches by finding a Calibre book Path with a book filename that is identical to the Zotero file attachment filename. It ignores everything in the Paths except the final filename. If you change the Calibre filename in any way, it will no longer match. Hence, the strong warning about not changing Calibre Author or Title prior to completing the ZMI update.

Calibre is renaming the files when I import them. That's why it didn't work. After your hint I tried this:
Imported some files into Calibre.
I then imported the associated files the Library directory of Calibre (they were already renamed) into Zotero.
Then I I followed the procedure I mentioned before and it worked.
Then I did something I didn't know it was possible - I did drag'n'drop some files from Calibre to Zotero. This has also worked and and it's much easier...
It doesn't bother me if it's working like this, I only found weird that I have first to import files into Calibre and then into Zotero and then back to Calibre. Is this the normal procedure?


By definition those 2 plug-ins are hardcoded to do only a single thing (ISBN and Word Count/Pages) by reading the .pdf text, and parsing it.


OK. But why are you mentioning this? Did you see the attached screenshots? I mentioned these plugins as an example - a user can have already other fields and columns which could theoretically then assigned to the Zotero fields. In the case of these plugins: ISBN and Pages. I had already other custom columns in my Calibre configuration before I start using Zotero which could then be assigned to Zotero fields. If there is no option to unite them then they can appear twice in the Calibre columns. To me it doesn't make sense but I'm happy with this version. Again, it's only a suggestion...

The fact that I'm getting 2 columns in the details view is because of Calibre or ZMI? Or why in the 2nd column only the values are showing up?

Last edited by excaliber; 06-23-2016 at 07:10 PM.
excaliber is offline   Reply With Quote
Reply

Tags
zotero

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Consolidate All Library Metadata (CALM) DaltonST Plugins 173 Today 02:36 AM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 362 04-25-2016 05:19 PM
[GUI Plugin] OPML Importer (RSS feeds from feedly/google reader) fem Plugins 14 07-19-2014 10:41 AM
Getting custom metadata field in GUI plugin SauliusP. Development 7 02-11-2012 01:31 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 09:37 PM.


MobileRead.com is a privately owned, operated and funded community.