[GUI Plugin] Zotero Metadata Importer

[nixsee]

Thank you very much for sharing your thoughts and experience. I definitely like the covers as well and was considering giving them up.

I assume that Zotero can open epubs (or any file format) in whatever reader you choose, no? Still, I appreciate that you'd want to simplify the process.

Wow, I hadn't noticed the lack of bulk editing, other than for Tags. That's bizarre. Mendeley can do it, but I'm not going to use Mendeley for obvious reasons.

I've become busy so this project of organizing my files has been put on the backburner for a little while, but I'm definitely leaning towards just using Calibre for everything. Thanks!

[DaltonST]

Version 1.0.65 - 2020-04-24 Regression testing and a general tune-up.

Suggestion: Add the 'Mark Books' action to the Calibre 'Context Menu' via 'Preferences > Toolbars & Menus > Context Menus'.



DaltonST

[t4exanadu]

I am having an issue getting ZMI to import all my articles as books. I am referring to step 2 here, adding of missing books to Calibre. My .csv has some 1500 rows, but when I add the books using ZMI it says 1097 rows are select, and then Calibre updates 230 books. Indeed, only 230 made it over. I took a look at the .csv, the file paths look good, and in Calibre all the fields appear to have been added (via step 1 which I have set to run at ZMI start).

What are some general steps for debugging issues with adding of missing books? Or have I completely misunderstood how this works?

Calibre 4.13 on Manjaro Linux, ZMI v1.0.65, Zotero 5.0.85.

[t4exanadu]

Originally Posted by t4exanadu

What are some general steps for debugging issues with adding of missing books? Or have I completely misunderstood how this works?

Before anyone else takes the time to answer that, I found the workflow for file import issues on page #1 (missed it because it's collapsed), and I'll have a look at that. I'll report back if nothing suggested in this thread works.

[DaltonST]

Be sure to set your options in the Options Tab to create books for HTML files. Calibre does not view HTML files as "books", so ZMI has special processing.

Also, the "Compare Collections" Tab also will tell you what is in Zotero but not Calibre per Collection in Zotero. It will update a ZCollection custom column automatically, and show it in the Calibre Tag Bar.

Oh...if your Zotero database is in the cloud, ZMI won't work for you. ZMI needs the actual SQLite database with its /storage/ folder underneath it.


DaltonST

[t4exanadu]

Ok, now I have an identifiable problem and some questions. I ran calibre-debug --gui (I'm using Linux) and then proceeded with "select single CSV" step. I'm seeing some errors:

Erroneous CSV File Attachment in Zotero file is being skipped: ERROR: Zotero file could not be copied to the temp directory: /home/andy/zot-temp/3Uagnv_zotero_ /home/andy/gdrive/Academic/Gallagher - 2001 - Philosophical Antecedents of Situated Cognition.pdf

Is the file being skipped because it can't be copied to the temp directory? I already confirmed permissions are correct on the temp directory, it should have no problems accessing it. So, I'm concluding there's another reason that is not clear from the error. That file path is valid, I can open the file at that location.

These files were not named by Zotero, perhaps that is the issue? Does ZMI expect a certain formatting for the file names? Which special characters are invalid (aside from those that aren't compatible with the filesystem, of course)?

[DaltonST]

ZMI impIorts standard Zotero export csv files with internal-to-Zotero files in the storage folder with ZKeys.

You are causing these errors.

Hover your mouse to read the many Tooltips.

[t4exanadu]

Originally Posted by DaltonST

ZMI impIorts standard Zotero export csv files with internal-to-Zotero files in the storage folder with ZKeys.

You are causing these errors.

Hover your mouse to read the many Tooltips.

I have read them. I'm using a CSV exported from Zotero. It has not been edited by hand. Each item (row) in the CSV has an entry in the Key column. Each row has an entry in the file attachment column. I have the option checked to add entries for books with no attachment, but that is irrelevant here, since every CSV item has a file attachment.

Maybe I misunderstand your words. Do the file attachments need to be in the same location as the csv and Zotero db file? It did import 230, I'm trying to figure out why it won't do the rest.

[t4exanadu]

OK, yes I found the issue.

[mahatma]

which version of ZMI compatible to calibre 3.48?

[DaltonST]

See the "sticky" thread at the top of Plugins: "'Old' Plugins for 'Old' Calibre Versions".

[bwrsandman]

Great Plugin!

One issue I ran into using the experimental python3 version of calibre is to do with the string and None comparisons done in zim_dialog.py

Linux-5.6.13-arch1-1-x86_64-with-glibc2.2.5 Linux ('64bit', 'ELF')
('Linux', '5.6.13-arch1-1', '#1 SMP PREEMPT Thu, 14 May 2020 06:52:53 +0000')
Python 3.8.3
Interface language: None
Successfully initialized third party plugins: Zotero Metadata Importer (1, 0, 65)
Traceback (most recent call last):
  File "calibre_plugins.zotero_metadata_importer.ui", line 137, in zmi_control
  File "calibre_plugins.zotero_metadata_importer.zmi_dialog", line 126, in __init__
TypeError: '>' not supported between instances of 'NoneType' and 'str'

This is easily fixed by replacing the two lines which do this:

# line 126
if prefs[k] is None or not prefs[k] > " ":
# line 131
if self.myparentprefs[k] is None or not self.myparentprefs[k] > " ":

[DaltonST]

Version 1.0.66 - 2020-07-09 Python 3 compatibility, especially for users who had never previously first used Python 2 ZMI.


DaltonST

[kobrick]

Hello,

I primarily use Calibre for my book management. Occasionally I have a book that I'd like to use in citations, and in that case I export to Zotero.
I would prefer to keep the book file in my Calibre storage location, and not copy it additionally to the Zotero storage.

When importing the RSI file to Zotero, I select "Link to files in original location" instead of "Copy files to the Zotero storage folder". However, the book is not linked, and attempting to open it from Zotero just gives an error "The attached file could not be found at the following path "/tmp/calibre...."".
If I choose to copy it to the Zotero storage, ZMI works correctly.

I am also able to manually link to the book file in the Calibre storage, but ideally the plugin would do this automatically.

Is this normal behavior, or am I doing something wrong?

Thanks all the work making this plugin!

[DaltonST]

When importing the RSI file to Zotero, I select "Link to files in original location" instead of "Copy files to the Zotero storage folder". However, the book is not linked, and attempting to open it from Zotero just gives an error "The attached file could not be found at the following path "/tmp/calibre...."".

See the L1 RIS Tag in the example RIS file shown below (created in Windows):

TY - BOOK        
AU - Ed Offley
DA - 2014-03-25        
DB - Calibre        
LA - eng   
L1 - C:\Users\[USERNAME]\AppData\Local\Temp\calibre_z2zfii\30.pdf
M2 - S:/Calibre/CalibreLibraryCodes/Ed Offley/The Burning Shore (30)/The Burning Shore - Ed Offley.pdf 
SN - 9780465080694        
SP - 538             
T1 - The Burning Shore                
ER -

ZMI must transform the Calibre absolute book path into a single temporary file name (hence your "/tmp/calibre....") that is both unique for a particular ZMI RIS export file and useful in reconciling Zotero to Calibre. The Calibre book ID (e.g. '30' as shown in the example) is unique by Calibre Library. In Calibre, searching on "id:30" will select the matching Calibre book for a Zotero item having a '30' in its filename ('30.pdf') or in the Extra field as described below.

Note that the temporary file shown in L1 will be automatically deleted no later than the very next Calibre shutdown.


Added: The use of a temporary directory to hold the book files being imported into Zotero was required to alleviate fatal security permissions errors in Zotero under Windows in 2016.


The M2 RIS Tag is (unless you change it) the Calibre absolute book path that is mapped by Zotero into the 'Extra' field. ZMI uses 'Extra' to retrieve the Zotero storage "KEY" to add to the Calibre book metadata of each respective book. That value is then used by ZMI to retrieve the Zotero Collection of each newly imported-to-Zotero book, and to determine what is and is not in Calibre based on Zotero Collection.


Added: I will look at creating a new pushbutton in ZMI for use when you ONLY want to Link to Calibre from Zotero. The original pushbutton will continue to be for ONLY importing the actual book file from Calibre into Zotero. There should be no OS security permissions for this use. I will simply set the L1 RIS Tag to be the same as the M2 Tag, which is to say the original absolute path of the Calibre book.


Added: Version 1.0.67 RIS File export: add new pushbutton (see the following post for details).

If I choose to copy it to the Zotero storage, ZMI works correctly.

ZMI always works correctly, since you are supposed to always choose to copy it to Zotero local storage. See the ZMI ToolTips and the documentation images attached to the Original Post of this same thread. You will find nothing about "linking" anywhere.






By the way, ZMI works only for Zotero-Desktop, and not for Zotero-Cloud. The Zotero Storage folder must be in the default local drive location created automatically by the Zotero-Desktop installer.



DaltonST