[File Type Plugin] Extract RIS Citations

[DaltonST]

[File Type Plugin] Extract RIS Citations


Summary:

Extract a Citation from RIS tags in .ris files and then update the new book's bibliographic metadata.

Descriptions:

• Citation: https://en.m.wikipedia.org/wiki/Citation
• RIS: https://en.m.wikipedia.org/wiki/RIS_(file_format)

Documentation:

Carefully review all of the attached images. Customization is required before use.


Companion Plugin:

Job Spy has a utility GUI Tool for RIS files, especially for those originating from Zotero. If you use Zotero, Job Spy is a must-have plugin. If you wish to import BIB or NBIB bibliography/citation files directly into Calibre from any source, such as PubMed for NBIB or BIB from many sources, Job Spy is also a must-have plugin. JS has Tools to convert, and then explode, multi-citation BIB and NBIB files into individual citation RIS files. This ERC plugin then processes the new RIS files in order to update the appropriate Calibre Metadata.

See: https://www.mobileread.com/forums/sh...d.php?t=269911



Minimum Calibre Version: 6.8.0


Version History:

Spoiler:

Version 1.0.15 - 10 Mar 2023 Miscellany. If missing AU and missing A1, existing A2 used for AU.
Version 1.0.14 - 26 Feb 2023 Change defaulted mappings of A1-A4 to #ris_primary_authors etc.
Version 1.0.13 - 25 Feb 2023 Change mappings of Custom Defined Tags C0-C9 & User Defined Tags U0-U5 for PubMed NBIB .nbib file metadata mappings and custom columns. Refer to the new Job Spy GUI Tool that converts NBIB to RIS, handing the converted RIS file over to this File Type Plugin, ERC.
Version 1.0.12 - 20 Feb 2023 Automatically correct malformed PY value (publication year); from BIB conversion to RIS.
Changed mapping of RIS Tag 'CP' from 'User Defined' to 'PubMedID', PMID (from MEDLINE).
Version 1.0.11 - 30 Jan 2023 A small percentage of .ris files have no AU (authors) tag, only A1 (primary authors) tags, causing the Calibre Author column to be "unknown". AU created for each A1 if the .ris file has zero AU tags.
Version 1.0.10 - 30 Jan 2023 Customization for DOIs, Tag "DO", enhanced to allow both a new Identifier of type "doi" and the RIS custom column for DOIs, #ris_doi, to be created or updated simultaneously. This facilitates matching DOIs for Zotero .pdf files imported by the Zotero Metadata Importer plugin with the DOIs for .ris files imported by this ERC plugin.
Version 1.0.9 - 26 Jan 2023 New 'Information' Dialog to notify user that they must use the Job Spy GUI Tool "RIS Citation File: Split Each RIS Tag Set into a Single RIS File" for the newly added .ris book, since the book contained multiple sets of RIS Tags, and each should be its own book.
Version 1.0.8 - 24 Jan 2023 Changes to accommodate the Job Spy GUI Tool "RIS Tags: Create Individual Custom Columns".
Version 1.0.7 - 22 Jan 2023 RIS Tags "Title" - TI & T1: Same as for new JS GUI Tool for RIS File Splitting/Exploding. Improved handling of multiple ISSN values in a single SN Tag.
Version 1.0.6 - 11 Jan 2023 Qt.core.
Version 1.0.5 - 14 Apr 2022 Qt6 compatibility. Minimum Calibre Version is 5.99.8.
Version 1.0.3 - 25 Dec 2020 Miscellany.
Version 1.0.2 - 13 Dec 2020 Calibre 5.7 compatibility. Minimum Version now 5.7.

Version 1.0.1 - 10 May 2020 DOI: improved handling if optionally mapped to Identifiers. New option: automatically resolve the DOI URL for the .ris just added as a 'book' to Calibre, and open it as a new tab in your default browser so that a desired document format (e.g. PDF) may be downloaded if desired.
Version 1.0.0 - 06 May 2020 Initial version of ERC.

[DaltonST]

Reserved for future use.

[DaltonST]

Version 1.0.1 - 10 May 2020 DOI: improved handling if optionally mapped to Identifiers. New option: automatically resolve the DOI URL for the .ris just added as a 'book' to Calibre, and open it as a new tab in your default browser so that a desired document format (e.g. PDF) may be downloaded if desired.


DaltonST

[DaltonST]

Importing Citations in XML, BibTeX, EndNote, or Mendelay Formats

Virtually all citation/bibliographic software has the means to export from their native file format to the RIS .ris format. If not, there is free conversion software available on the web to do so. So, this "Extract RIS Citation" file-type plugin can (at least indirectly) import .xml, .bib, .enw, and other bibliographic file formats.

Note that Zotero can be used in a "minimalist" fashion as an all-purpose "translator" for the above purpose. It will import all of the above formats, and then export them in RIS format.

If you use Zotero "fully", then you can use the "Zotero Metadata Importer" GUI plugin to more comprehensively import your Zotero .pdf files and their associated Zotero metadata into Calibre. The "Extract RIS Citation" file-type plugin is unnecessary if you use ZMI.



DaltonST

[DaltonST]

Version 1.0.2 - 13 Dec 2020 Calibre 5.7 compatibility. Minimum Version now 5.7.



DaltonST

[DaltonST]

Version 1.0.3 - 25 Dec 2020 Miscellany.


DaltonST

[DaltonST]

Qt6 compatibility. Minimum Calibre Version is 5.99.3.


DaltonST

[DaltonST]

New version using only Qt6 & Python 3 has been attached here:

https://www.mobileread.com/forums/sh...42&postcount=7


DaltonST

[DaltonST]

Version 1.0.5 - 14 Apr 2022 Qt6 compatibility. Minimum Calibre Version is 5.99.8.

[DaltonST]

DaltonSt, many thanks for this plugin, and for the Extract RIS Citations (my comment is mainly about this last one, since the thread for it is closed on mobileread). My case is as follows: I’m trying to import metadata records from Zotero into Calibre – metadata only, I don’t care about file attachments, since I don’t have any such attachments in Zotero. I’ve tried importing a CSV via ZMI for this task, which works great, except it leaves out significant Zotero fields (or fields I’m interested in), like “Abstract”. I then tried importing a RIS file via Extract RIS Citations, which has the advantage of importing all RIS tags, including the Abstract (AB) one, – BUT, the problem is that Extract RIS Citations only works with RIS files that contain one single record/citation. My RIS files exported from Zotero contain several records/citations per file, and your plugin only extracts the first record in the file. Is it possible for you to code the Extract RIS Citations to handle such multi-record RIS files, in order to extract all records/citations in a RIS file, not just the first one? Thanks.

A single RIS file may include many unrelated sets of RIS Tags. To add a single new RIS "book" into Calibre, only the very first Set of RIS Tags is used to perform the update of the RIS file type.

If you want multiple books added from a single RIS File, simply break the long file into multiple pieces of one (1) set of RIS Tags per piece. Then, drag-and-drop each "piece" separately onto the Calibre GUI to create a new "book" from that piece, including the metadata extracted from the RIS Tags per your configuration.

The RIS Specification allows only one (1) Citation per set of RIS Tags.

The ZMI plugin was not designed to import Citations, but only a .pdf file along with its Metadata from Zotero. Zotero is for Citations. Calibre is for "books" (defined very loosely).

Perhaps Zotero has a standard (or plugin) function to create a new blank .pdf file containing all of the Zotero Metadata for a particular Item. You could then export that to a .CSV file, and import it into Calibre using ZMI. The citations would be in the text of the .pdf file just referenced.


DaltonST

[stefan5]

Originally Posted by DaltonST

If you want multiple books added from a single RIS File, simply break the long file into multiple pieces of one (1) set of RIS Tags per piece. Then, drag-and-drop each "piece" separately onto the Calibre GUI to create a new "book" from that piece, including the metadata extracted from the RIS Tags per your configuration.

I thought about splitting the RIS file into multiple chunks and then importing each one individually into Calibre, but that would add an extra step and would fragment my multi-record RIS files into several thousand unique-record files. The RIS files exported from Zotero may contain hundreds, even thousands of records/citations/“books” per file...

The RIS Specification allows only one (1) Citation per set of RIS Tags.

This may be so, although many academic databases (Ebsco, ProQuest, etc.) do not follow this specification, exporting the metadata for several titles into one single multi-record RIS file (which can be then imported without issues into Zotero).

Thanks, anyway. I’m thinking now of using ZMI, and then your CSV tool from JS+ to import/update the CSV fields that ZMI leaves out.

[DaltonST]

This may be so, although many academic databases (Ebsco, ProQuest, etc.) do not follow this specification, exporting the metadata for several titles into one single multi-record RIS file (which can be then imported without issues into Zotero).

Therein lies the root-cause misunderstanding of "who does what" in Calibre.

The "[File Type Plugin] Extract RIS Citations" is...a 'File Type Plugin'. Where does the 'File Type' come from? You, when you drag-and-drop it onto the Calibre GUI. Calibre then creates a single (1) book, automatically, from the file that you just told it to by dropping it. Then, Calibre automatically invokes all File Type Plugins for that particular File Type (a.k.a. File Extension). This plugin is currently the only File Type Plugin for .ris File Types, so it gets invoked. It then does what it does to the (now pre-existing) .ris "book".

In short, this plugin cannot create a new book. It works on only pre-existing (albeit only a millisecond old) books with a format of .ris.

This File Type Plugin was intended to import individual .ris files based on a specific DOI downloaded from, for example, sources such as ascopubs.org/doi/ and not at all from Zotero.

What you are asking for is a brand new GUI Plugin, not File Type Plugin, that can do what Zotero does as regards importing .ris files into it from many academic databases.

Added: I am going to add a new Job Spy GUI Tool called "Split RIS File into Component RIS Tag Sets", which will populate the Calibre Auto-Add folder with the resulting multiple single-set .ris files created from a single selected multi-set .ris file.


DaltonST

[stefan5]

Originally Posted by DaltonST

Added: I am going to add a new Job Spy GUI Tool called "Split RIS File into Component RIS Tag Sets", which will populate the Calibre Auto-Add folder with the resulting multiple single-set .ris files created from a single selected multi-set .ris file.
DaltonST

That would be fantastic, thanks!

[DaltonST]

Check the JS Thread for a beta test version.

DaltonST

[stefan5]

Wow, that was fast Initial test with a RIS file w/ 200 records completed in about 2 minutes, with perfect splitting. I will test a more demanding RIS file of 2000+ records tomorrow. Huge thanks for this!