[GUI Plugin] Extract EPUB Metadata

[ovillo]

Recover fanfic metadata from EPUBs

If FanFicFare can't reach a story anymore: AO3's down, the fic got deleted, or you got the epub someplace else (browser extension, a friend, an old backup), you can end up with a Calibre entry with no usable metadata. This plugin tries to fix that by reading the metadata straight out of the EPUB itself.

You choose how it writes the data back: straight into Calibre's standard fields, into your own custom columns, or as raw data ready for FanFicFare's own metadata pipeline to process normally.

It also has a separate action for anthology EPUBs (multiple fics bundled together) that recovers each individual story's own completion status, since the bundle's own status usually just tells you if the whole collection is finished or not.

Tested sources include AO3 and fanfiction.net EPUBs made with FanFicFare, AO3, ao3downloader, fichub, and WebToEpub, including some Wayback copies. Collection and converted PDF support is limited.

Requirements: Calibre 6.0 or newer.

Install: Download the single `extract_epub_metadata-v1.0.0.zip` attachment to this post. In Calibre, use Preferences → Plugins → Load plugin from file.

See the README for configuration, examples, and limitations:


Source and license: https://github.com/wordsandpics/Extract-epub-metadata — MIT.
Issues: https://github.com/wordsandpics/Extr...etadata/issues
Version history: https://github.com/wordsandpics/Extr...n/CHANGELOG.md
Feedback and suggestions very welcome.

[ovillo]

Updated with a few bugfixes and tweaks. Feedback welcome!

Extract epub metadata 0.2.1 beta

[ShoeQueen]

I'm very excited to see this being developed! This is my exact scenario, I have many downloaded fics that I need only metadata for, but fanficfare now struggles with AO3s cloudflare so much that I rarely get it to work.

I'm a Windows 11 user, and not a programmer, but I did give it a try. I couldn't get the plugin to load. It gave an invalid error:

calibre.customize.InvalidPlugin: The plugin in 'C:\\Users\\XXXXX\\Downloads\\Extract-epub-metadata-0.2.1-beta.zip' is invalid. It does not contain a top-level __init__.py file

Not sure if this feedback really "helps", but I figured I'd send it on anyway.

Thank you and all the developers for working on this!

[ovillo]

Thanks for trying it!

That error is actually my fault. I did not attach the plugin file to the latest release, and that file is GitHub's automatic source download, not the plugin zip.

I've fixed it: on the release page, download the zip under Assets named extract_epub_metadata-v0.2.1-beta.zip (not "Source code"), then Preferences → Plugins → Load plugin from file.

I've attached the file here too.

Sorry for the trouble!

[ovillo]

V. 0.3.0 Beta

New:
- AO3 works recovered from the Internet Archive Wayback Machine, saved with either WebToEpub or the Save as eBook browser extension. The story URL is set to the real AO3 link, and the Wayback snapshot URL is kept as a new "Wayback Machine URL" field.
- AO3 PDF downloads converted to EPUB (e.g. in Calibre): tags, series, stats, title, author and summary are read from AO3's preface page.
- Series is now recovered from AO3 downloads, work pages and PDFs, and can be written to Calibre's Series field or a custom column.

Fixed
- Old `download.archiveofourown.org` story links are recognized.
- Calibre's "Unknown" placeholder author is no longer treated as a real name.
- "Update standard Calibre metadata" read the wrong series value.
- A finished work could be marked In-Progress when AO3's stats had no spaces between them.
- For Wayback copies, FanFicFare now gets the real AO3 URL (as a `url` identifier) instead of the Wayback one.

[ShoeQueen]

This plugin is fantastic, and so quick! I gone through about 2,000 epubs, and it's been working great!

I originally setup Calibre by using Easy Fanfic Library https://codeberg.org/the_bi_ballerin...Fanfic-Library because coding is like black magic to me... completely mystifying.

The only issues I have a note on I haven't been able to get the Collections to properly fill in. This could totally be user error, as I have only played around with it for a little while. If you haven't included it yet, then pay me no mind. I completely thrilled with what you have done!

But even with that little bit not working on my end, I have to say that this is AMAZING! I'm not any kind of coder, so if I was able to figure out how to use it, then you already have a pretty great plugin.

You have saved me hundreds of hours of getting frustrated with AO3 and Cloudflare. My Calibre is now actually searchable so I don't have to rifle through thousands of stories to find a fic.

Keep up the good work! When you do an official release it's going to absolutely blow everyone's mind!

Thank you! And if you ever need someone to test something specific, lemme know. I got a lot more fics to get through, so there's always fresh epubs to work on.

[ovillo]

I'm so happy to hear someone else is getting use out of this. I basically made it for myself, and hoped it would be useful.

Bt collections, do you mean anthologies downloaded with FFF? If so, yea, haven't attempted much except extracting completion status. What would you like/expect to happen?

And if you ever run into a fic that does not work properly, send it to me, either here or through github and I can look into what's going wrong and try to fix it!

[ovillo]

updated to v1.0

[ShoeQueen]

Originally Posted by ovillo

Bt collections, do you mean anthologies downloaded with FFF? If so, yea, haven't attempted much except extracting completion status. What would you like/expect to happen?

Sorry for not getting back to you sooner! What I meant was on AO3 the Collections that people use to gather stories of a particular type that pull from multiple fandoms and/or authors. Like "Best of (pairing or fandom)" or used when doing Gift exchanges, or challenges.

I have a column in FFF for those. Mind you this (in my mind) is the least important information to be gathered - but was a nice addition when setting it up. And again, this could be user error on my part.

Either way, this plug-in is awesome!

[madame_newbie]

Hi! Thanks so much for making this plugin! I was wondering if it's possible for you to do a version where it can extract metadata from a txt file? Or ones from fanfiction.net?

I recently downloaded a massive archive from archive.org, and while I am able to extract url via fanficfare, it requires me to either update them via fanficfare which for 100k books is a bit much or I have to do it manually.

thanks so much!

[ovillo]

ff.net is somewhat supported- it kinda depends on how you downloaded the story (fichub, for example, works quite well). Other sources and formats depend on how the data is presented and whether the plugin knows how to find it.

The easiest thing is if you can share some samples of stories that are not working properlu, then I can examine them and figure out if there's a way to find the metadata. if you can attach the files here or send them trhough wetransfer/megauplaod/google drive or similar service, I'll gladly take a look.

[madame_newbie]

Hi I've attached here the file I got from archive.org. I can extract the url via fanficfare but the issue is that I'd have to add it in manually one by one and some of these fics are of course no longer available on ff.net itself.

I am not sure what the original uploader used to scrape these fics, however.

[ovillo]

thanks, that helps! some questions:
- did you get it from archive.org- from the original page itself? or from someone else's archive?
- could you send over 1 or 2 more files?

the more information/samples I have the greater the chance I can do something that works for all (or most) files and not just one