[GUI Plugin] Zotero Metadata Importer

[snow-raven]

Originally Posted by DaltonST

Having said that, see this Job Spy plug-in ...

https://www.mobileread.com/forums/sh...42118#poststop

DaltonST

for this plugin and for the Job Spy suggestion! I now have my references in calibre & the ability to add any Zotero metadata I choose. I can't tell you how happy it makes me to see all of those little document covers and to be able to glance at the abstract in the Book Details. It's so much better than my old file folder structure, and it's tools like these that made it possible. It might even be better than going to the library.

[snow-raven]

Hello,
I've recently updated both calibre and the plugin and just experienced this error when adding metadata to the selected books. I've tried restarting and re-exporting my csv file. Can anyone point me at a fix?
Thanks!

calibre, version 4.3.0
ERROR: Unhandled exception: <b>HTMLParseError</b>:EOF in middle of entity or char ref, at line 1, column 34

calibre 4.3 embedded-python: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '7', '6.1.7601')
Python 2.7.16+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 6, 1) && Job Spy (1, 0, 167) && Zotero Metadata Importer (1, 0, 60) && BookFusion Plugin (0, 2, 3) && View Manager (1, 4, 3)
Traceback (most recent call last):
File "calibre_plugins.zotero_metadata_importer.zmi_dial og", line 1675, in automatically_update_calibre_metadata_control
File "calibre_plugins.zotero_metadata_importer.zmi_dial og", line 1893, in update_zotero_custom_columns
File "site-packages\calibre\utils\html2text.py", line 22, in html2text
File "site-packages\html2text\__init__.py", line 151, in handle
File "site-packages\html2text\__init__.py", line 163, in close
File "HTMLParser.py", line 121, in close
File "HTMLParser.py", line 216, in goahead
File "HTMLParser.py", line 124, in error
HTMLParseError: EOF in middle of entity or char ref, at line 1, column 34

[DaltonST]

Your Zotero export csv has a column with html that cannot be parsed without errors due to its complexity. Keep that column header but clear its data and rerun.

[snow-raven]

Thank you. It turned out to be an "&" in a manual tag. Surprisingly, the "ñ" in another tag was fine and the Html italics in some of the journal article titles were not the culprits.

It took me a while to find the problem record, as the column number referenced did not match the position of the column in the export file. Is there a way that I can see the actual column order in ZMI next time?

[DaltonST]

Originally Posted by snow-raven

It took me a while to find the problem record, as the column number referenced did not match the position of the column in the export file. Is there a way that I can see the actual column order in ZMI next time?

Run Calibre in Debug Mode (ctl+shift+r or right-click the Preferences toolbar icon and click Restart in Debug). When you process the input .csv from Zotero, each row is printed to the Debug Log as it is processed, not to mention other data that is of use when debugging your data.


Remember that the top row/first column line in your .csv is row 0/column 0.




DaltonST

[DaltonST]

Version 1.0.61 - 2019-11-15 Miscellaneous Technical Changes.

Improved debugging mode information available to Users of ZMI for their Zotero exported .csv data.

Debugging mode in Calibre is initiated by either ctrl+shift+r or via mouse from the toolbar Preferences > click the down arrow > Restart in Debugging Mode.

Remember to periodically compress/defragment your Calibre metadata database via Library > Library Maintenance > Check Library. Your metadata.db file will grow unnecessarily large due to metadata changes for your imported Zotero books.

Also periodically back up your configuration etc. via Library > Export All Calibre Data.




DaltonST

[DaltonST]

Version 1.0.62 - 2020-01-02 Technical changes after Python 3.8 testing with Calibre 4.99.2


DaltonST

[nixsee]

I have read through this whole thread, and realize it is mostly for bugs. But I'm hoping that someone can give me a brief answer to my question regarding use cases for ZMI.

I've been using Calibre for years for my ebook collection and just discovered Zotero which I've started using more for academic papers. I have now discovered this plugin and am curious about integrating the two systems.

I like Calibre for the various plugins, custom columns, metadata with reviews, good integration with devices, conversion between formats, etc... I like Zotero specifically for the ease of adding academic papers, taking notes, collections, and indexed search (which seems less useful for a book). It seems that integrating the two with ZMI makes duplicates of all of the files and just generally adds complexity. So, I'm wondering when, why and how do people tend to use one or the other, and why would they integrate rather than just keep them separate (or stick with one)?

I'm not at all trying to be contrary - I genuinely would like to know what benefits, complements and workflows I'm obviously not seeing.

Thanks!

[DaltonST]

Calibre and Zotero are not "integrated"; they are "interfaced". Huge difference.

Most of the 2500+ users heavily use only the Zotero-to-Calibre interface.

The Calibre ZMI RIS interface file to Zotero is used by people who want to start using Zotero after first using Calibre, or want to add some Calibre ebooks to Zotero and continue using both.



DaltonST

[nixsee]

Thanks for the quick reply and clarification on integrated vs interfaced - it is what I intended.

Given that most only use ZMI to move from Zotero to Calibre, what features does Calibre offer that they find lacking in Zotero?

[eliasy]

Hi Dalton,

Thanks for the awesome plugin.

I am using the RIS generated file to send books in the direction Calibre -> Zotero.
However, I am having a problem with the author field.

It is being parsed as 'Long Name', where we have 'First Name Middle Names Last Name'.

Since Zotero works preferentially with the last name of the author, (and betterbibtex uses the last name to create citations, e.g., Shannon1948), would you consider adapting the AU field in the RIS file to 'Last Name, First Name Middle Names'?

It would be interesting to also make name abbreviations homogenous (using a metric distance, e.g., Manhattan distance) as a option. For example, we could have Harry C. Smith, Harry Smith and Harry Charles Smith becoming Smith, H. C. Of course, it would be hard on this to avoid collisions (Harry Charles Smith and Herman Common Smith). But since it would be optional, the user could decide if he/she wants to have this risk.

Thanks for your time.

[DaltonST]

Calibre is not English-only. However, I can add the option to use Author_Sort in lieu of Author in the RIS export file to Zotero.

name	                                sort	
Jenna Elizabeth Johnson	                Johnson, Jenna Elizabeth

DaltonST

[DaltonST]

Version 1.0.64 - 2020-04-02 RIS tag "AU": {authors} OR {author_sort}; ToolTips for RIS Export configuration revised accordingly.

Please read the RIS configuration ToolTips before using {author_sort}.

Note that {author_sort} can be customized via Tweaks such that the end-results may not be suitable for exporting to your RIS export file for Zotero.

Use 'Bulk Metadata Edit' to update 'Author Sort' en masse using your most recent customization settings.


DaltonST

[eliasy]

Owa! That was fast!

Thanks a lot Dalton.

I tested it, and apparently it is working just fine.

Best!

[snow-raven]

Originally Posted by nixsee

Thanks for the quick reply and clarification on integrated vs interfaced - it is what I intended.

Given that most only use ZMI to move from Zotero to Calibre, what features does Calibre offer that they find lacking in Zotero?

Hi, nixsee! Since you asked, the biggest draw for me with Calibre is that I'm fairly visual in my organizational style. I can find the reference I'm looking for most quickly if I can see the cover. I use pretty extensive tagging and have some virtual libraries sorted by subject, but that last find cue is usually a visual grab from the cover layout. If I have to think of a title or author name, it takes me much longer.

I also appreciate that I can store epubs and pdfs equally easily, and Calibre's epub reader is easy to use.

Plus, Calibre is generally more powerful & better maintained. For example, it still makes me insane that I can't bulk edit Zotero fields without coding. Bulk editing is Computing 101. They've been promising to add that feature for years with no effect. I can also store my Calbre library in my dropbox and access it from multiple computers and devices. Zotero doesn't support that & has dire warnings against it. You can, however, subscribe to them to store your library. I'd rather give my support to a program that I see actively improving - Calibre. If Calibre generated citations, I'd drop Zotero completely.