[GUI Plugin] FanFicFare

[JimmXinu]

Yes, you should be able to use replace_metadata to modify the series name by watching for seriesUrl or author.

Something like:

add_to_replace_metadata:
 series=>^.*$=>Other&&seriesUrl=>http://test1\.com\?seriesid=1

[ownedbycats]

Thank you

Originally Posted by ownedbycats

Just for documentation, I just updated a fanfic where I had some text highlighted with Calibre's new annotation feature. (Storing annotations inside the ePub is disabled.) It added the new chapter and didn't touch the annotation whatsoever.

Also, it seems that glitch I had with the reading position getting shifted upon ePub update no longer happens.

No lost/shifted annotations, either. I suspect it would happen if annotations were embedded and the update is set to overwrite the ePub. I think Kovid mentioned that annotations are stored in a few places though so it might be able to recover.

[JimmXinu]

New Version Released

Version 3.24.0 - 15 Oct 2020
- fiction.live - handle api returning non-int values for votes PR #568, thanks muchtea
- adapter_scribblehubcom: Include author notes/news in chapters unless in exclude_notes - thanks, Epicpkmn11!
- archive.hpfanfictalk.com -> fanfictalk.com plus site changes for same.
- Change ponyfictionarchive.net to https by default
- Making eFiction series name/desc collection work with more sites.
- Apply CLI --json-meta to --list to see series name/desc.
- Fix adapter_storiesonlinenet login. Closes #562
- Add AO3 Series 'status'.
- Don't save cover image size for dedup_img_files - Calibre might replace it. Also fixes #561
- Better GUI for series anthology
- Adding eFiction series parsing attempt to base_adapter.
- ensure_text plugin-example.ini for py3
- New site: www.the-sietch.com XenForo2 site with weird URL scheme. Closes #430
- Add get_urls_from_page() and get_series_from_page() to adapters, add support. AO3 and TtH only so far.
- Add include_author_notes option (defaults on) to adapter_royalroadcom Closes #556
- adapter_fictionlive: Handle stories with missing contentRating, storyStatus & support beta.fictionlive.com domain, thanks HazelSh!
- Add setting `remove_tags:script,style` replacing script remove hardcode, adding style remove. Closes #553.
- New Site: novelupdates.cc (possible transition dest for wuxiaworld.co?) - Thanks ElminsterAU!
- Fix for adapter_wuxiaworldsite not setting authorUrl.
- Add dedup_img_files option(default:false). #550
- wuxiaworld.co and novelupdates.cc don't author links after all?

[ownedbycats]

Thanks for all the work on this

[bigwoof]

Originally Posted by JimmXinu

New Version Released

Version 3.24.0 - 15 Oct 2020
- fiction.live - handle api returning non-int values for votes PR #568, thanks muchtea
- adapter_scribblehubcom: Include author notes/news in chapters unless in exclude_notes - thanks, Epicpkmn11!
- archive.hpfanfictalk.com -> fanfictalk.com plus site changes for same.
- Change ponyfictionarchive.net to https by default
- Making eFiction series name/desc collection work with more sites.
- Apply CLI --json-meta to --list to see series name/desc.
- Fix adapter_storiesonlinenet login. Closes #562
- Add AO3 Series 'status'.
- Don't save cover image size for dedup_img_files - Calibre might replace it. Also fixes #561
- Better GUI for series anthology
- Adding eFiction series parsing attempt to base_adapter.
- ensure_text plugin-example.ini for py3
- New site: www.the-sietch.com XenForo2 site with weird URL scheme. Closes #430
- Add get_urls_from_page() and get_series_from_page() to adapters, add support. AO3 and TtH only so far.
- Add include_author_notes option (defaults on) to adapter_royalroadcom Closes #556
- adapter_fictionlive: Handle stories with missing contentRating, storyStatus & support beta.fictionlive.com domain, thanks HazelSh!
- Add setting `remove_tags:script,style` replacing script remove hardcode, adding style remove. Closes #553.
- New Site: novelupdates.cc (possible transition dest for wuxiaworld.co?) - Thanks ElminsterAU!
- Fix for adapter_wuxiaworldsite not setting authorUrl.
- Add dedup_img_files option(default:false). #550
- wuxiaworld.co and novelupdates.cc don't author links after all?

Thanks! very interested to try the series support feature! :-)

[DavidTC]

So, a feature request:

I play around with metadata parsing a lot, over and over again, and I obviously have a saved metadata column. And when I'm doing that, I change the option to 'Update Calibre Metadata from Saved Metadata Column'.

And then I'm done, and I don't remember to change it back to 'Always Update EPUB if New Chapters', and I do the email update. And of course, it doesn't update the existing stories. I did tell it to start popping up the confirmation box, but I still will forget to change it, or miss it.

And of course, with the email update, it has now marked those emails as read. So I have to go to my email, mark them all as read (paying attention to how many emails I have to go back) and then run the entire damn thing again. So this is sorta unique extra annoyance with email.

So, can we perhaps always update existing stories on email?

I understand this ability to ignore existing books when updating individual URLs, or page-scrapped URLs, you might already have some of the books and know it, and don't want to hit the servers for no reason. We all sometimes lose our place when adding from tabs or paging through stories or whatever and copy a URL twice.

But email? What's the Use Case here? Just...randomly marking my email as read?

So, I'd like to ask for a an 'override default existing story option' checkbox on the Email Settings tab.

Or, and this might be more consistent, you could have three different defaults on the first tab, one for 'Default when run on an existing Calibre book', one for 'Default when manually adding URLs directly or via page scrap', and one for 'Default when getting URLs from email'. (Or maybe you only need two and the last two could be merged.)

[DavidTC]

To expand that a bit:

1) When people try to update a book they know exists, by choosing it in Calibre and running update, they can want different things to happen, and a default there makes sense. They might want to add new chapters (Probably 99% of the use), they might to rewrite the entire book because something went wrong, they might want to update just the metadata.

Although, interesting, they don't ever want to Skip the book...that doesn't make sense, to choose a book in the list just to skip it. So every choice except Skip makes sense there, both as a presented option as a default.

2) Whereas, when they provide a list of URLs, they always want the newest version of that book in their library. There's no person who says 'I want the newest version if I don't have it, but I'd like to regenerate the metadata if I already do'. Huh?

So the only other option that make sense is 'Skip', because they don't want to increase the server load if they accidentally paste the same thing twice, or if added a couple of book and then decided to scrap the author's entire page.

Conclusion:
The different forms of updating a book are actually different things. 'updating a book by selecting it', and 'getting a bunch of URLs that might include existing books in it'. They can work the same, but outside of the core functionality, they go off in completely different directions of what people might be trying to do.

The first thing needs the ability to select a default option from the list.

The second just needs a checkbox meaning 'I'm trying to be polite to the server and I don't recall what tabs in this giant browser mess I've already added.' for pasted stuff. (Really, a 'Do not try to update these pasted URL if the books have been updated in the last few hours' checkbox might make more sense...it's even less server load.)

And email just...needs to update, always. There's no scenario where someone has a story in their library, and got an email saying it had changed, and they would like FFF to...simply mark that email as read and not update the story.

[ownedbycats]

Originally Posted by DavidTC

There's no person who says 'I want the newest version if I don't have it, but I'd like to regenerate the metadata if I already do'. Huh?

Counterpoint: I have a few AO3 series where the authors add new stories to the middle of the series. I have FFF use 'update always' to fix the series numbering on existing books.

[DavidTC]

Originally Posted by ownedbycats

Counterpoint: I have a few AO3 series where the authors add new stories to the middle of the series. I have FFF use 'update always' to fix the series numbering on existing books.

But you do that by seeing a new book comes in for one of those series, so you filter on that series in Calibre, select all the books, and update them that way, right?

Selecting the books in Calibre does need a default option. It's the other way, providing a list of URLs, I was saying doesn't, because people always want 'Make sure I have the newest version of this URLs' when they specifically hand FFF a URL, there is no other use case.

[DavidTC]

Or do you navigate to the series page and paste it into the scrape page thing, updating all the books in the series that way? Which...could be possible, I guess. But there's not a clickable link to a series in Calibre, so... you have to click on the fic link to open it up in a web browser, and then go to the series once it does, and then copy the URL and paste it in.

And...if the series is on multiple pages the hassle is even bigger.

So doing it that way seems unlikely vs. just clicking on the series in Calibre and hitting Ctrl+A and then Update.

But I mean, if people are actually doing that...okay, whatever.

But the emailed URLS should always default to updating, and pasted URLs should have a separate specific default from Calibre-selected-books update. (Or not let you set a default for pasted ones, either...even if they let you pick one in the prompt box.)

[ownedbycats]

Hitting Ctrl+A and updating wouldn't add the new fics...

If I'm adding it, it's because I see it on the website. Or I could just copy the URL from the title page of a story from the same series.

I grab the series URL and put it in "Get Story URLs from web page," which then feeds into the get story URLs dialog. This way I get both the new story and update all the old ones in one step.

[MrKonan]

Updating or downloading stories from storiesonlinenet now gives you a prompt for login/password to login to download the story. Log says "failed to login for url xxxx with username xxxx"

[JimmXinu]

@DavidTC, I will be looking at your suggestions when I get a chance.

But I can assure you from long experience that assuming the way *you* do things is the way *everybody* does things is a mistake. Doesn't matter who you are or what the things are.

[JimmXinu]

Originally Posted by MrKonan

Updating or downloading stories from storiesonlinenet now gives you a prompt for login/password to login to download the story. Log says "failed to login for url xxxx with username xxxx"

Are you using the latest version of FFF? Because I don't see that.

[MrKonan]

Originally Posted by JimmXinu

Are you using the latest version of FFF? Because I don't see that.

Yes. v3.24.0

Here's a snippet from debug log.

FFF: DEBUG: 2020-10-17 09:26:35,451: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(104): Will now login to URL (https://storiesonline.net/sol-secure/login.php) as (xxxxxx@gmail.com)
FFF: DEBUG: 2020-10-17 09:26:35,452: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1152): #####################################
pagecache(GET) MISS: https://storiesonline.net/sol-secure/login.php
FFF: DEBUG: 2020-10-17 09:26:35,840: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1010): Encoding:utf8
FFF: DEBUG: 2020-10-17 09:26:35,856: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1080): #####################################
pagecache(POST) MISS: https://login.wlpc.com/index.php?cmd...om&v=igGc(rest are deleted..)
FFF: DEBUG: 2020-10-17 09:26:36,001: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1010): Encoding:utf8
FFF: DEBUG: 2020-10-17 09:26:36,010: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1080): #####################################
pagecache(POST) MISS: https://login.wlpc.com/index.php?cmd=Log In&email=xxxxxx@gmail.com&password=XXXXXXXX&v=igGc (rest are deleted..)?Accept=text/html,*/*&Content-type=application/x-www-form-urlencoded
FFF: DEBUG: 2020-10-17 09:26:37,542: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1010): Encoding:utf8
FFF: INFO: 2020-10-17 09:26:37,543: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(174): Failed to login to URL https://storiesonline.net/sol-secure/login.php as xxxxx@gmail.com
FFF: WARNING: 2020-10-17 09:26:37,544: calibre_plugins.fanficfare_plugin.fff_plugin(1072) : Login Failed, Need Username/Password.