[GUI Plugin] Zotero Metadata Importer

[DaltonST]

Also, I cannot do anything without the CSV file that you used to create the debug log. I always need that too. They always go together. Thanks.

[DaltonST]

@petzi:

I think this is it: "Meine E-Books"

Calibredb is dropping the double quotes around the path for some reason, causing this error message: /E-Books/Calibre/Zotero-Test not found

ZMI gives calibredb all of the parameters with double quotes around them since they have spaces, so I need to understand why it still fails. So, I need to see the OS path of the file.

Can you please copy and past the exact path obtained using your OS of that Calibre Library in your next post?

In the meantime, for an easy and fast test, change the space in "Meine E-Books" to and underscore "_", and run the test again. CSV and debug log.

Thanks.

[DaltonST]

Version 1.0.13 - 2016-06-28 Changes.

@petzi: I think this will fix the fundamental problem that you have been having.



DaltonST

[petzi]

Yep, this worked! Even with the blank in the path name.

(I did the test after I had done the library maintenance.)

I also tried it then with a larger group of files (the 109 files I had used before) and again: it worked!

(For the 109 files I had to wait a pretty long time with spinning cursor, so that I didn't know if the programme has crashed. Maybe a kind of progress indicator would be helpful?)

When I will have time I will inspect the result and give you feedback. As I told you I have to follow up other obligations and additionally I will be some time without internet connection.

[DaltonST]

Version 1.0.14 - 2016-06-30 Major Enhancements.


ZMI is a week old today. Thanks to the long-time users of Zotero, with this version it becomes highly usable.

See the image below.

Remember to hover your mouse over each area and widget of the window so that you may read the ToolTips.

The ToolTips are the User Guide.



DaltonST

[DaltonST]

Version 1.0.15 - 2016-06-30 New: Progress Dialog with ability to cancel; miscellaneous tweaks; ToolTip enhancements.

If you are a new Calibre user due to the availability of ZMI, be aware that your Calibre metadata database can in general get very large over time, but especially given the use of ZMI to add your huge number of Zotero books, their metadata into custom columns, and then updating standard and custom Calibre metadata from those Zotero custom columns.

Fortunately, it can easily be compressed/defragmented/vacuumed by this menu path: Library Icon > Library Maintenance > Check Library.

Another suggestion: do not run a PC temporary file cleaner while ZMI is running (or Calibre even with ZMI not running). Not good. Bad things happen.



DaltonST

[DaltonST]

Version 1.0.17 - 2016-07-01 Many small enhancements to increase the efficiency of the ZMI 'process'.



DaltonST

[DaltonST]

Version 1.0.19 - 2016-07-03 Enhancements to increase the efficiency of the ZMI 'process'.


Everyone should upgrade to this version.



DaltonST

[DaltonST]

Version 1.0.23 - 2016-07-07 New: 'Auto-Select Multiple CSV Files' then Auto-Process all of them simultaneously. Afterwards, archive or delete the original CSV files.


See the attached image.



DaltonST

[DaltonST]

Version 1.0.24 - 2016-07-12 Miscellaneous enhancements.




DaltonST

[petzi]

I am back again…

I saw that you were very busy during the relatively short time span of 14 days. When we stopped conversation we had ZMI version 1.0.13 and now we have 1.0.24!

We stopped the conversation when the import with ZMI worked for me. In the meanwhile I had time to check the imported files and it appears that they are a mess. But the biggest problem: I couldn't find any pattern.

• The only field which is filled in correctly is the field 'size' with the Zotero file included.
• Many times the author is not known ('Unknown"), sometimes not.
• Several times the year (field 'published') is included correctly, several times not (about 50/50)
• Sometimes the title is correct, many times not.
• When tags are added, they are added wrongly, from whatever source I do not know.
• In only one case the publisher is added correctly
• Besides size, generally only the fields title, author and tags are filled in, even the .csv-file has all the other information in the correct fields. All the Z-fields remain empty.

Actually I do not know how to proceed to give you more specific information. Maybe the best way would be to do it one case by one case. I think/hope that there will be just one wrong program logic which is responsible for the overall mess. So what I propose: I will give you some specific information from the .csv file and the resulting entry in my calibre database. So you can compare and find out what went wrong.

What do you think? Would this be a feasible procedure? Or do you have another (more effective) idea? – Or maybe I did something wrong and I am responsible for the database mess myself?

[DaltonST]

@petzi:

ZMI works perfectly 100% of the time for me.

I suspect that you just continued where you left off with invalid test data using an obsolete version of ZMI to add it to Calibre. Your "mess" should simply be deleted.

Read all of the ToolTips in both Tabs of ZMI.

Create a new Calibre Library for Zotero, and using the very latest ZMI version with the proper ZMI Options set, import all of your Zotero books and their metadata.

Use Auto-Step Mode with Auto-Select All CSV Files. Activate them in Options.


DaltonST

[petzi]

Thanks, it worked!

The reason of my problem was not that I used an old version but that I used "Single-Step Mode" and stopped after the first button "… then Add Missing Books to Calibre".

With the different setting "Auto-Step Mode" and "Auto-Select All CSV Files" turned on there were different steps included and the "mess" in my database was cleaned up! It worked too in "Single-Step Mode" when I continued the process and pressed the buttons of the next group "Update Caliber Metadata Using Zotero Metadata" as well.

A wonderful plugin!

[DaltonST]

Citavi Users:

If you wish to add your Citavi PDFs and their metadata into Calibre in an automated fashion, then you have two ZMI-related options.

Option 1: Install Zotero, and import your Citavi data and files into Zotero. Then, export your "Citavi-Zotero" data into a standard Zotero CSV file. Use ZMI as normal for adding Zotero books and their metadata to Calibre.

Option 2: Do not install Zotero. Create an export file in Citavi from Table View with the columns that map 1:1 to Zotero CSV columns. Use ZMI to add your Citavi books and their metadata to Calibre using that pseudo-Zotero CSV file. However, Option 2 requires critical extra work on your part, as follows:

[A] The CSV column names in the very first row must be changed to be identical to their counterparts in Zotero. For example, in Citavi there is a column "DOI Name", but in Zotero it is simply "DOI".

[B] The very first CSV column must have a unique identifier that ties the PDF file in the CSV "File Attachments" column to that row of the CSV file. Zotero actually embeds that unique identifier, "key", in the OS path of the PDF files. Citavi does not. Therefore, you must create a user-defined field in Citavi, populate it with a unique identifier for each PDF file, and map that user-defined field to the Citavi export file. Be advised that "unique" means "unique in the world", and not simply "unique within this CSV file". I suggest using a spreadsheet function to generate a many-digit alphanumeric random value, such as is done for UUIDs.

Best of luck.



DaltonST

[DaltonST]

Version 1.0.26 - 2016-07-15 New: Compare Zotero Database to Calibre Database to find Zotero books missing from Calibre.

See the attached image.

I have tested this with both Standalone Zotero using Firefox, and Firefox Zotero. Sometimes Zotero puts an exclusive lock on its database. If that occurs, ZMI will tell you, and will advise you to simply exit from the Zotero software so that it releases its lock.

Version 1.0.27 - 2016-07-15 Miscellaneous enhancements; New Calibre Identifier of 'zcollection' is updated for books with their matching Zotero collection name automatically whenever the Compare function is executed for their Zotero collection.



DaltonST