[GUI Plugin] FanFicFare

[JimmXinu]

New Test Version Posted

2025-03-10
- Add remove_empty_p option, usually for AO3/OTW. #1177

This has been asked for more than once and I never put it in, partly because I thought (and still think) it's something AO3 should have addressed. Right now, I just want to be able to fix something.

Let me know if you think it's too narrow or too broad, but I will expect references to story chapters showing that.

## Some sites/authors/stories (notably AO3/OTW) add empty p tags where
## they aren't intended by the author during document upload and not
## all authors know how/take the time to fix it.  This feature removes
## all "empty" <p> tags, ie, those containing only whitespace or <br>
## tags.  Literally: <p[^>]*>\s*(<br/>)*\s*</p>
#remove_empty_p: false

[Droewyn]

Is anyone else having issues with uploading updates from email? Five of my fics updated today, and when I used FFF to download them it gave me the "none of these need to be updated" message. I verified that the version on ao3 has more chapters than the ones in Calibre.

Not sure if this has anything to do with the current, dreaded 525 error.

[ownedbycats]

Thank you! I have to manually edit those before.

[booster-t]

Originally Posted by steltek

Yeah, I've noticed it too.

I've been playing around with FFF settings in personal.ini, but haven't found anything that definitively helps with it yet though.

Sometimes, commenting out the "is_adult:true" setting for A03 in personal.ini lets it work. But, it isn't consistent.

This fix is inconsistent as noted. What's so frustrating is that you can download the whole story using A03's "show entire story" and then download. The main drawback to that is Calibre does not update the date of update :-) and there are no chapter numbers.

[ownedbycats]

Is it possible to get open_pages to work without use_browser_cache_only? The download worked that time, so I couldn't tell.

[JimmXinu]

Originally Posted by ownedbycats

Is it possible to get open_pages to work without use_browser_cache_only? The download worked that time, so I couldn't tell.

No.

Without use_browser_cache_only, FFF tries to download directly when a page isn't found in cache. open_pages_in_browser was a later addition.

[fmk007]

Is it possible to add official AO3 mirror URL https://archive.transformativeworks.org to FFF?

Today opening story links in AO3 is very slow and lead very often to an error "SSL handshake failed". Also FFF get a timeout often.

The AO3 mirror site works as usual with the same requests, but the URL is not recognized with FFF.

Example:
AO3 (https://archiveofourown.org/works/48576721 -> very slow, sometimes error
Mirror: https://archive.transformativeworks.org/works/48576721 -> fast with no problems

[JimmXinu]

I don't recall hearing of archive.transformativeworks.org before.

The current adapter code allows several domains based on what was allowed by their ssl cert at the time, but always normalizes to archiveofourown.org:

    @classmethod
    def getAcceptDomains(cls):
        return ['archiveofourown.org',
                'archiveofourown.com',
                'archiveofourown.net',
                'download.archiveofourown.org',
                'download.archiveofourown.com',
                'download.archiveofourown.net',
                'ao3.org',
                ]

But you appear to be correct that it is an official mirror judging by the FAQ What URLs can I use to access the Archive?

The question is whether those different domains point to different systems, or are just aliases to the same system. DNS does give different IPs for archiveofourown.org vs archive.transformativeworks.org, but given today's cloud systems, how much does that mean?

Another problem: I was not able to login on archive.transformativeworks.org -- it still tried to log me in on the usual archiveofourown.org domain. So even if it works better, you can't use it for stories/features requiring login.

Yet another issue: the list of most recently updated stories I get back from the two domains don't match. archive.transformativeworks.org appears to be several stories out of date. That argues against it just being an alias, but how out of date?

However, this is worth trying out.

(Very) Test Version

Attached is a test version that adds archive.transformativeworks.org as if it were a new, unrelated site.

I do not expect it to work with logging in, and it does not use your existing AO3 settings.

This is purely so people can test and report if it's less error prone for them.

If it is, I will look into how using archive.transformativeworks.org might be integrated into the AO3 adapter.

[ownedbycats]

Originally Posted by JimmXinu

If it is, I will look into how using archive.transformativeworks.org might be integrated into the AO3 adapter.

I wonder if it could become an automatic fallback for AO3 failures. It's probably too much work, especially since AO3 is relatively stable except for recent.

[ThreeApplesTall]

Hey, long-time lurker, first-time poster here!

Just a quick FYI – I'm on macOS and kept getting the "HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428)" error when trying to use the browser cache workaround, despite setting the limit to -1.

While trying to problem-solve, I saw some mentions of macOS possibly blocking calibre from opening browser tabs in background jobs. I'm not 100% sure if that's what the issue was, but I managed to get the openers_by_scheme tweak to work that's been mentioned here before, and that seems to have fixed it for me. I just had to pass it the full path to the firefox executable file.

openers_by_scheme = {'http*': '/Applications/Firefox.app/Contents/MacOS/firefox %u'}

Since then, I've had no more error messages with loading any fics, and while it's of course still considerably slower than it was before the whole cloudflare situation, it now opens all the browser tabs it needs without any issues and is a lot faster and more reliable than it was without using the cache.

Just in case this ends up helping someone else!

[JimmXinu]

Thanks! I've linked to this from the issue over on github.

[kayydotmurr]

Forgive me, I'm new to all of this, but with the ao3 errors going on, I'm wondering if there is a way to just update the URL fetched from inside the story into my URL column and not search the internet for updates right now? The excel spreadsheet I'm moving data over from has a URL column, and I'm thinking that will be easier to match up than titles where I may have typed it slightly differently (I downloaded the import list plugin but I've got errors there too haha so I'd like to compare what data moved over.)

[icecold]

I'm trying to use ignore_chapter_url_list to download a fanfic without two specific chapters and I'm not sure what's going wrong. I've tried shortening to thread url to QQ/threads/24696/ but that didn't seem to work either

[https://XXX/threads/hell-won%%E2%%80%%99t-have-me-heaven-won%%E2%%80%%99t-take-me-so-i-guess-i%%E2%%80%%99ll-just-gaslight-everyone-highschool-dxd-oc.24696/]
ignore_chapter_url_list:
 https://XXX/posts/7611265/
 https://XXX/posts/7612003/

Moderator Notice
MobileRead doesn't like NSFW links. Obfuscating this little may also be pushing it.

[JimmXinu]

Originally Posted by kayydotmurr

Forgive me, I'm new to all of this, but with the ao3 errors going on, I'm wondering if there is a way to just update the URL fetched from inside the story into my URL column and not search the internet for updates right now? The excel spreadsheet I'm moving data over from has a URL column, and I'm thinking that will be easier to match up than titles where I may have typed it slightly differently (I downloaded the import list plugin but I've got errors there too haha so I'd like to compare what data moved over.)

Not really.

FFF has a feature "Get Story URLs from Selected Books" that can get the story URL from inside the epub. But it returns the url from identifiers field first if it's a valid story URL. And it just gives you a list of urls, it can't populate a column for you.

When you download the epubs using FFF in Calibre, FFF will populate the identifiers field with the story URL and the Publisher field with the site name. It seems like either of those would serve your purpose.

[JimmXinu]

Originally Posted by icecold

I'm trying to use ignore_chapter_url_list to download a fanfic without two specific chapters and I'm not sure what's going wrong. I've tried shortening to thread url to QQ/threads/24696/ but that didn't seem to work either

The INI file format uses % as a control character. You need to either escape them all to %% or use a shortened story URL. I suspect you shortened too much, use [https://QQ/threads/24696/] Except with the forums... domain name instead of QQ. I did test that it still works.

Note that MobileRead doesn't like NSFW links. I removed them for you.

(No idea why the [mod] tag didn't work...)