[GUI Plugin] FanFicFare

[culytera]

Originally Posted by JimmXinu

So it's really only a problem once per work. How often are you seeing this? My understanding is that orphaned works are quite rare.

This might be possible if it's something people see regularly. But FFF will only have the past author info if you already had include_logpage on during the previous update. Customizable titlepage means FFF can't read it back.

Sometimes I change the look of the title pages (and the cover) so I do a redownload of my entire collection via "Overwrite Always" since there's no "Change Title Page Only and Keep Current Metadata" update mode. Not ideal but it is what it is. I only noticed this problem with a few older works that had been orphaned by their authors when I updated their title pages, but it's my mistake for not enabling include_logpage. If the orphaned work has a logpage with the previous author name before turning into orphan_account, would that have its own value name for use in titlepage_start or maybe a comma-separated list?

[SeaWeed]

I’m confused about how to update stories from AO3. Does FFF automatically update the fics I’m subscribed to? How about the fics I’ve bookmarked? Or do I have to manually update fics with the “Update Existing Fanfiction Books” option?

Thanks.

[culytera]

Originally Posted by SeaWeed

Does FFF automatically update the fics I’m subscribed to? How about the fics I’ve bookmarked?

You could set FFF to fetch updates via email for the works you're subscribed to, but I personally don't do that. (FanFicFare settings -> Email Settings tab)

If you don't want to setup updates via email, you'll have to select the fic on Calibre and have FFF update it. It doesn't automatically update bookmarked works.

Originally Posted by SeaWeed

Or do I have to manually update fics with the “Update Existing Fanfiction Books” option?

Yes.

You could make a custom column called #status that shows whether the fic is In-Progress or Complete, then have a Virtual Library in Calibre that filters the list to just In-Progress works so you can easily find whichever incomplete work you want to update.

To set this up:

1. Create a custom column with the lookup name "status", column heading "Status", and the column type "Text, but with a fixed set of permitted values"
2. In FanFicFare's settings windows, go to the Custom Columns tab
3. Select "Status" as the metadata value for your custom column #status
4. Click "Virtual library" beside Calibre's search bar and select "Create Virtual Library"
5. Set the "Virtual library name" to whatever you want
6. Set the search expression to #status:"=In-Progress" and click Ok

[JimmXinu]

Originally Posted by culytera

Sometimes I change the look of the title pages (and the cover) so I do a redownload of my entire collection via "Overwrite Always" since there's no "Change Title Page Only and Keep Current Metadata" update mode. Not ideal but it is what it is. I only noticed this problem with a few older works that had been orphaned by their authors when I updated their title pages, but it's my mistake for not enabling include_logpage. If the orphaned work has a logpage with the previous author name before turning into orphan_account, would that have its own value name for use in titlepage_start or maybe a comma-separated list?

You're doing "Overwrite Always"? That's a pretty intensive way to update things... And honestly, it says "Overwrite" right on it.

And that's worse for server traffic than trying to update an entire library. Unless you have a few hundred stories max (and even then batch the updates), this is not a use case I want to encourage.

The New Only options are the only direct method I can think of for FFF to partially update metadata.

Currently, the only metadata exposed from logpage data is the calculated words_added entry. There is no other current way to use the log data.

You might consider "Pass Calibre Columns into FFF" option (bottom of FFF config personal.ini tab). Then you have the current Calibre column values to work with.

Originally Posted by SeaWeed

I’m confused about how to update stories from AO3. Does FFF automatically update the fics I’m subscribed to? How about the fics I’ve bookmarked? Or do I have to manually update fics with the “Update Existing Fanfiction Books” option?

No, FFF doesn't do automatic updates.

See FAQs:

• Why shouldn't I just select my entire library and use FanFicFare plugin to check for updates?
• How do I setup FanFicFare to read my story update notification emails for me?

Also, honestly, this may be a poor time to start using FFF for AO3.

Direct access on AO3 is working for me this morning, but we've spent the last several weeks with most users having to use workarounds to download from AO3 at all. See recent pages of this thread.

[culytera]

Originally Posted by JimmXinu

You're doing "Overwrite Always"? That's a pretty intensive way to update things... And honestly, it says "Overwrite" right on it.

And that's worse for server traffic than trying to update an entire library. Unless you have a few hundred stories max (and even then batch the updates), this is not a use case I want to encourage.

The New Only options are the only direct method I can think of for FFF to partially update metadata.

I redownload them in batches and I have the cache settings enabled so I thought that would somewhat mitigate the server requests if FFF is still using cached versions.

Upon reading https://github.com/JimmXinu/FanFicFare/wiki/UpdateModes again I realize "Update EPUB Always" would've been better for my use-case since "This option is frequently used to update the title page within the EPUB." But the epub's cover only changes when "Update Calibre Metadata?" is also checked and the tooltip says it only retrieves the metadata from the website. What if I only made changes to the titlepage format and the output css and didn't need to have the entire EPUB redownloaded for a work that hasn't been cached? Like "Update Calibre Metadata from Saved Metadata Column" but it's to update the EPUB's title page, stylesheet css, and embedded cover.

Originally Posted by JimmXinu

You might consider "Pass Calibre Columns into FFF" option (bottom of FFF config personal.ini tab). Then you have the current Calibre column values to work with.

Yes that works. I replaced authorHTML with an a tag that uses authorUrl and calibre_std_authors so the old author name is still used.

Partial code for those wondering:

<h1 class="title"><a href="${storyUrl}">${title}</a> by <a class='authorlink' href='${authorUrl}'>${calibre_std_authors}</a></h1>

[JimmXinu]

Originally Posted by culytera

What if I only made changes to the titlepage format and the output css and didn't need to have the entire EPUB redownloaded for a work that hasn't been cached? Like "Update Calibre Metadata from Saved Metadata Column" but it's to update the EPUB's title page, stylesheet css, and embedded cover.

I've considered this at least twice over the years that I recall, the time I can find is here and eventually ended with:

Originally Posted by JimmXinu

I have spent waaay more time (including another hour today) than this corner case can justify trying to fit this into the existing system in way that both works 'as I expect to' and without breaking anything else. I give up.

It's not like there isn't already a way to do it, it's just not as efficient. Use 'Update EPUB always', or don't; it's your choice.

Admittedly, orphaned (or deleted) works can't just use 'Update EPUB always' equivalently, but it is an even smaller corner case.

[marshie250]

Originally Posted by culytera

For works on AO3 whose authors changed to "orphan account", how do I maintain the old author on the page itself? For metadata, I have "Author(s)" set to New Only, but I'm not sure how to keep that value in the epub itself. Though it would be much better if it only maintained the old author value if the new one is "orphan_account".

I include these lines in my personal.ini:

#orphan
[https://archiveofourown.org/works/########]
add_to_replace_metadata:
author=>orphan_account=>authornamehere

[ownedbycats]

Originally Posted by JimmXinu

So it's really only a problem once per work. How often are you seeing this? My understanding is that orphaned works are quite rare.

This might be possible if it's something people see regularly. But FFF will only have the past author info if you already had include_logpage on during the previous update. Customizable titlepage means FFF can't read it back.

Generally I see it happen because the cover image doens't automatically regenerate. I just add a storyurl replace_metadata with the old author in this case.

[vorm]

Originally Posted by Tee_Kay

So, i had an # berofe "use_browser_cache_only:true", my mistake there.
It seems to work now, just even slower than before.

Is there a way to "automate" this? i.e. add a line in Personal.ini to make it automaticly close the tabs after a fic is done? Don't want to always close everything manually.
Tested it with 2 fics, with one chapter each, at the same time. Both opened, and then from both each chapters and a Json html site.
Something about groups. I think its just if the autor is in a group.

Don't know what would happen if i try to batch download 10 fics with, lets say 10 chapters each.
Don't want the browser, Flaresolverr or windows to crash.

Originally Posted by JimmXinu

Not that I'm aware of. There may be a browser extension out there somewhere that will auto close tabs?

The whole BrowserCache thing is an ugly hack to work around the ever increasing levels of automated blocking in response to the ever increasing levels of automated access going on.

So I came here to ask about this Spacebattles error I keep getting ('NoneType' object has no attribute 'find_all'), probably because of the UI changes, and ended up in this talk… so I may have vibe-coded a hack around this (because god it is annoying to try to back up a large number of fics). Basically I created a plugin that intercepts the tab open command from Calibre and instead sends it to a Chrome extension that opens the page in the background and later closes the tab. You could probably do something way less hacky if you want to add this natively to the plugin. Either way, it worked for me for at least the 500+ fics I used it to back up/update, and since it looks like this is a pain point for more than just me, I decided I should spread the love and contribute a bit.

Here is a zip with the plugin and extension source code. There is a readme of how to install/use it also. Enjoy.

https://file.kiwi/1d152650#7P8nmKBA0ngjB6LCW9dwqw

[hanjisung]

I really apologize, as I'm not sure how to troubleshoot this or even best describe the issue in a way that can actually be addressed.

FanficFare was working very well as far as I know, up until a couple updates ago. I've been hoping the issue would correct as updates come, but I'm still having issues, and since others don't seem to be having the same problems as I do, it must be something more like user error... I just don't know in what way, as I didn't change the .ini or the way I do things, to my memory.

My first issue is this:
1) I loaded on several PDFs from my external hard drive onto my Calibre, and I tried to extract the links from the PDFs (all selected) with FFF, as I've done in the past. "Get URLs from Selected Books" But it keeps coming back "No Story URLs found in selected books." Are my PDFs just too old, that somehow FFF can't work with them the same way it did my other PDFs?

2) When I try to "Download from URLs" it says the URLs I've provided do not need to be downloaded, despite the fact that I do not have them in Calibre (some, as EPUBs, some at all.)

Status
Title
Author
Comment
URL
Error
Unknown
Unknown
HTTP Error in FFF '403 Client Error: Forbidden for url: https://archiveofourown.gay/works/18...=true'(403)
https://archiveofourown.org/works/18628708

^ the log I get back, for example.

I allow FFF to login with my username and password in my .ini by the way.

eta: Wait I'm going to try disabling uBlock Origin.
eta2: No, that didn't do anything for me...

Sorry but I don't understand this... I tried turning the uBlock Origin extension off, and also adding:
https://archiveofourown.org
https://archiveofourown.gay
to 'Trusted Sites' on the extension as well. Neither attempt fixed the issue.

I am on Linux, and I have Snapd Firefox (not flatpak.)

But I swear FFF has downloaded for me since my move over to this system and software :/

[culytera]

Originally Posted by JimmXinu

I've considered this at least twice over the years that I recall, the time I can find is here and eventually ended with: [post]

Admittedly, orphaned (or deleted) works can't just use 'Update EPUB always' equivalently, but it is an even smaller corner case.

Yes it's unfortunate but I suppose it can't be helped.

Originally Posted by marshie250

I include these lines in my personal.ini:

#orphan
[https://archiveofourown.org/works/########]
add_to_replace_metadata:
author=>orphan_account=>authornamehere

I didn't have a log of the original author names and I forgot the authors of a few orphaned works so this wouldn't work for those. Since I was doing a titlepage remake of my entire library, I only noticed the orphaned ones afterwards.

I'll just need to be more mindful the next time I remake the titlepage for my entire library again.

Originally Posted by hanjisung

I really apologize, as I'm not sure how to troubleshoot this or even best describe the issue in a way that can actually be addressed.

FanficFare was working very well as far as I know, up until a couple updates ago. I've been hoping the issue would correct as updates come, but I'm still having issues, and since others don't seem to be having the same problems as I do, it must be something more like user error... I just don't know in what way, as I didn't change the .ini or the way I do things, to my memory.

My first issue is this:
1) I loaded on several PDFs from my external hard drive onto my Calibre, and I tried to extract the links from the PDFs (all selected) with FFF, as I've done in the past. "Get URLs from Selected Books" But it keeps coming back "No Story URLs found in selected books." Are my PDFs just too old, that somehow FFF can't work with them the same way it did my other PDFs?

2) When I try to "Download from URLs" it says the URLs I've provided do not need to be downloaded, despite the fact that I do not have them in Calibre (some, as EPUBs, some at all.)

Does your .ini still have use_archiveofourown_gay:true? Your log shows it's checking for the work in the .gay domain instead of .org.

HTTP Error in FFF '403 Client Error: Forbidden for url: https://archiveofourown.gay/works/18...7(403'(403)

[hanjisung]

Does your .ini still have use_archiveofourown_gay:true? Your log shows it's checking for the work in the .gay domain instead of .org.

Yes: "use_archiveofourown_gay:true"

I thought that was for the better? To somehow cut down on the traffic to .org?

[culytera]

Originally Posted by hanjisung

Yes: "use_archiveofourown_gay:true"

I thought that was for the better? To somehow cut down on the traffic to .org?

There were discussions a few pages back about archiveofourown_gay not working anymore for getting downloads so people have switched to using the cache settings.

Try these instead:

[defaults]
## To find the cache2 folder's location for the correct profile on Firefox: navigate to about:profiles on your address bar and click "Open Folder" on Local Directory under the profile that says "This is the profile in use and it cannot be deleted."
browser_cache_path:C:\Users\Username\AppData\Local\Mozilla\Firefox\Profiles\profile_name\cache2

[archiveofourown.org]
use_browser_cache:true
use_browser_cache_only:true
## Change browser_cache_age_limit to -1 if 6.0 isn't enough
browser_cache_age_limit:6.0
open_pages_in_browser:true
## Adjust slow_down_sleep_time to however long you want
slow_down_sleep_time:2

[hanjisung]

Originally Posted by culytera

Try these instead:

Ah! You're amazing!

Because I'm on Linux Ubuntu, I had to dig around a bit to find where my cache2 path actually was. In case anyone else is using that system, idk if this is the correct path for YOU, but here was mine, in case it helps:

/home/[username]/snap/firefox/common/.cache/mozilla/firefox/[lettersandnumbers.default]/cache2

Thank you <3 Your help is sincerely appreciated. I would've been heartbroken the next time I tried to dl a bunch of fics to read.


Still having the issue of not being able to get URLs from PDFs, if anyone has advice on that front.

[culytera]

Originally Posted by hanjisung

Still having the issue of not being able to get URLs from PDFs, if anyone has advice on that front.

Enable the setting in FanFicFare under the Basic tab: "Search inside ebooks for Story URL?"

AFAIK PDFs aren't supported so you'll need to convert them to EPUB in Calibre first so FFF can detect links.