[GUI Plugin] Extract People & Other Metadata

[DaltonST]

[GUI Plugin] Extract People & Other Metadata

Summary: EPOM uses your personally-constructed 'Python Regular Expressions' to search the text of an ebook and extract metadata from it.

Documentation: The EPOM 'User Guide' is comprised of all of its ToolTips plus any images and related files attached below.

Requires Minimum Calibre Version: 6.11.0

Other Useful Calibre Plugins to Consider:

• [GUI Plugin] CalibreSpy
• [GUI Plugin] View Manager

Version History:

Spoiler:

Version 1.0.0 - 2023-01-06 Initial release.

[DaltonST]

For Future Use

[arpeggioaccele]

I installed Extract People & Other Metadata through calibre user plugins and used it to extract Ao3 links from the epubs I downloaded, though I'm not sure if I did it the best way. (I have been intentionally avoiding fanficfare) Thought it would be good to share if anyone else needed my janky solution.

I couldn't figure out how to remove a set string at the beginning and end of the text so here's the solution I made.

I made a custom column named link.
I set up three #link custom column extractors:
To identify work links I used the keyword "Posted originally on the.+$" I wanted to make I got the right link, sometimes there are other work links in the file.
I filtered the result to just the work id number and s/ "s\/\d+"
Then I in tweaks I added

REMOVE_CHARACTERS=.<>[]()
CALIBRE_TEMPLATE_LANGUAGE_BUILTIN=re(1, 'Posted originally on the Archive of Our Own at ', '')
CALIBRE_TEMPLATE_LANGUAGE_BUILTIN=re(1, 'Posted originally on the Archive of Our Ownhttp://archiveofourownorg/ at ', '')
CALIBRE_TEMPLATE_LANGUAGE_BUILTIN=re(1, 's/','http://archiveofourown.org/works/')

which removes the extraneous period and original keyword and adds back the website url after by replacing the /s. When you don't have a fulltext index, the raw html needs to be filtered differently.

To identify series links it was easier just assume there was only one and use the keyword "http:\/\/archiveofourown.org\/series\/.+$"
There was no period after to remove, but I filtered for the properly formatted url just in case.
I only turned this extractor on for works I combined with epubmerge.

For fanfiction.net works that I downloaded with ficlab I used the keyword "based on content retrieved from.+$"
I filtered for the story id and s/ "s\/\d+"
I replaced s/ with the website url after removing extraneous characters.

REMOVE_CHARACTERS=.
CALIBRE_TEMPLATE_LANGUAGE_BUILTIN=re(1, 'based on content retrieved from ', '')
CALIBRE_TEMPLATE_LANGUAGE_BUILTIN=re(1, 's/', 'https://www.fanfiction.net/s/')

And now I have a link in book details that I cannot click as a link. I probably should have made it an identifier or something lol, but it works for my purpose of exporting to a spreadsheet.

[arpeggioaccele]

I now realize this can be done through fanficfare or mass search and replace to find urls in books grab metadata, and it correctly make the url an identifier... oops.
https://www.mobileread.com/forums/sh...ntifiers+links
https://www.mobileread.com/forums/sh...nk#post4320727
https://www.mobileread.com/forums/sh...ntifiers+links

[Comfy.n]

I wish I had tried this amazing plugin before using the Job Spy similar tool!

[danmeian]

This is an awesome tool. Is there a way to bulk extract info from multiple ebooks, instead of doing one at a time?
If it's necessary to do 1 at a time, is there a way to avoid the result screen popping up (where it shows the updated book, and you need to click back to the home screen, then do another search to get to where you originally was)?
Thank you!

[Comfy.n]

edit: it's ok now

[danmeian]

@Comfy.n: Awesome, tysm!! This has been super useful. A small issue with this is that when I do multiple books without using FTS index, the result pops up 1 by 1. Consequentially, only the last extracted book get marked. This makes it a bit inconvenient when going back and forth between different search pages. This aside, the function definitely works.

Question: Is there anyway I can use this to extract the first 3-4 paragraphs from an epub?

Background info: I'm trying to generate a cover from the first page of an epub. Calibre's default "set cover from book" doesn't seem to work too well, so my plan is to
1. Use EPOM to extract the first 3-4 paragraphs from the epub (into custom column #first_pars)
2. Use Generate Cover to create a cover with text from #first_pars

[Comfy.n]

Well I've used EPOM just for extracting translators and original titles, using the FTS index. That was not too challenging. In your case it would be better if Dalton could help, but it's been almost a year he's away from MR, unfortunately. Or maybe some regex power user.

I don't see an easy way to detect the exact beginning of the text, given the ebooks' structure variations, however you could try something like this

- set the tweak MAXIMUM_LENGTH_TO_ACCEPT= to a large value
- then populate the #first-pars column using regex to match, say, the first 1000 chars in the book

[killo3967]

This is a very usefull tool, thanks for it.

I have a problen that is, after extract a translator to a custom column, the return value is "Translator: Diorki" for example. How can i get from the ":" to the end?

[Comfy.n]

I had an issue with a MOBI format: UnboundLocalError: cannot access local variable 'is_finished' where it is not associated with a value

Upon conversion to EPUB, plugin worked fine, thankfully.

[msel]

Hello, I don't know if I understand the plugin correct: I've epub books which have this tag in the content.opf (e. g.: <dc:source>https://www.heise.de/blog/CQRS-als-Grundlage-fuer-moderne-flexible-und-skalierbare-Anwendungsarchitektur-10275526.html?view=print</dc:source>)
I want to extract the website address and add it to custom text column #source. How can I do this with this plugin? Or is there another possibility (e. g. identifiers) to extract such a tag from a ebook file?
Greetings from the Münsterland, Maria

[Comfy.n]

Originally Posted by msel

Hello, I don't know if I understand the plugin correct: I've epub books which have this tag in the content.opf (e. g.: <dc:source>https://www.heise.de/blog/CQRS-als-Grundlage-fuer-moderne-flexible-und-skalierbare-Anwendungsarchitektur-10275526.html?view=print</dc:source>)
I want to extract the website address and add it to custom text column #source. How can I do this with this plugin? Or is there another possibility (e. g. identifiers) to extract such a tag from a ebook file?
Greetings from the Münsterland, Maria

I think you might have better luck in the Plugin ideas thread, as DaltonST is long-time MIA.

From looking at this tooltip, I gather EPOM is capable of extracting text, not OPF HTML content:

[msel]

Hello Comfy.n,

thank you - i had the same idea that it won't works with the content.opf. I'll add the idea to the thread.

Greetings from the Münsterland, Maria!

[Maitresinh]

What a wonderfull plugin !

I'd like to add all the authors from an anthology, with that expression

^([\p{L}\.'\- ]*\p{Lu}{2,}[\p{L}\.'\- ]*)$ i ^(.*)$


...but i allways get a "no custum column is active...nothing to do".

What am i doing wrong ?

I've check the case at the left.