[GUI Plugin] FanFicFare

[miajoshua]

HTTP Error in FFF '525 Server Error: for url: https://archiveofourown.org/works/58...79759'(525) URL:'https://archiveofourown.org/works/58229944?view_adult=true'(525)
https://archiveofourown.org/works/58229944

Read on the last page that several people have encountered this error with AO3 but managed a work around by manipulating the "is adult=" line by either turning it off (adding # in front) or inputing "false".

Neither one worked.

I had no problems when I just login to the same story with my web browser (Chrome/MacOSX) so not quite sure what's going on.

Been trying all night with no success. Any ideas?

Thanks!

[Amalthia]

Originally Posted by JimmXinu

New Test Version Posted

2025-03-10
- Add remove_empty_p option, usually for AO3/OTW. #1177

This has been asked for more than once and I never put it in, partly because I thought (and still think) it's something AO3 should have addressed. Right now, I just want to be able to fix something.

Let me know if you think it's too narrow or too broad, but I will expect references to story chapters showing that.

## Some sites/authors/stories (notably AO3/OTW) add empty p tags where
## they aren't intended by the author during document upload and not
## all authors know how/take the time to fix it.  This feature removes
## all "empty" <p> tags, ie, those containing only whitespace or <br>
## tags.  Literally: <p[^>]*>\s*(<br/>)*\s*</p>
#remove_empty_p: false

Thank you so much for making this happen! I don't always have time to check stories to remove the empty <p> tags and if I can add this to my settings it'll save me so much time.

I really appreciate all your efforts to make this plugin amazing and useful.

Amalthia

[kayydotmurr]

Originally Posted by JimmXinu

Not really.

FFF has a feature "Get Story URLs from Selected Books" that can get the story URL from inside the epub. But it returns the url from identifiers field first if it's a valid story URL. And it just gives you a list of urls, it can't populate a column for you.

When you download the epubs using FFF in Calibre, FFF will populate the identifiers field with the story URL and the Publisher field with the site name. It seems like either of those would serve your purpose.

Yeah last night I was just getting the URLs from the document and then pasting them into my columns one by one, because the FFF plugin was not fetching the metadata, I was getting that same 525 error on ao3 everyone else has been. And I already HAVE the files, I’m just new to managing them in calibre. I had them all in a folder on my cloud before and kept a spreadsheet with a couple pieces of info on it. So I’ve got partially filled out metadata for almost 500 epubs. I’ll just work on them manually over the weekend.

[JimmXinu]

Originally Posted by miajoshua

Been trying all night with no success. Any ideas?

Thanks!

IMHO, the site is just randomly broken. People are waving dead chickens that sometimes coincide with it working.

[Kiloee]

Any data that could be provided to show if it is indeed random?

Because I am stumped with the same error. While being perfectly able to read all stories I attempted to download in the browser, the download just shows the 525 Server Error.

I can even open up additional new stories on Ao3, so it ain't just my cache doing it thing as far as I can tell.

[JimmXinu]

AO3, like all of the internet now it seems, uses a service called CloudFlare. Which I don't blame them for--any and every public site can become the target of a DDoS at any time for no known reason. And it has happened to them.

CF is, fundamentally, a big business service specifically to block automated access to web sites. FFF is, fundamentally, a way to automate access to web sites.

We've been dealing with this for years now on fanfiction.net and I ran out of energy to fight it directly long ago.

AO3 has publicly reported that they are having server problems and using a higher level of CF blocking. Hopefully things will work better again next month.

In the meantime, there's the Browser Cache feature, also discussed here repeatedly.

There is another possibility I'll post about separately.

[JimmXinu]

(Very) Test Version Attached

Nobody said anything about the version I posted that did archive.transformativeworks.org as a separate site.

This version has a switch to force the usual archiveofourown.org adapter to instead use archive.transformativeworks.org for all network requests:

[archiveofourown.org]
use_archive_transformativeworks_org:true

Note that it cannot log in because log in doesn't work on archive.transformativeworks.org.

Other than the actual network reqs, this version still uses and records the normal AO3 domain.

This seems to work for me. I'd like to hear other peoples results.

[Thorne3]

Thank you JimmXinu for making something that I have enjoyed every single day for almost a decade.

I have a problem however that might be my misunderstanding.
Thank you for any help you're able to give.

Include_images:false does not prevent images being downloaded and bloating the epub file.

Context:
example url:
https://FQQ/threads/naruto-waifu-den-commissioned.22459/

Personal ini:
[defaults]
is_adult:true
include_images:false

[<fqq domain>]
include_images:false

Moderator Notice
Please don't post NSFW links on Mobile Read.

[JimmXinu]

Please don't post NSFW links on Mobile Read. I've removed them for you this time.

The include_images:true directive in defaults.ini is under [epub]. [epub] as a format, takes precedence over both [defaults] and [<site>] sections in personal.ini. There's a wiki page about FFF INI including precedence.

You can either use:

[<fqq domain>:epub]
include_images:false
## for one site, or

## for all downloads.
[overrides]
include_images:false

[GloverEggs]

I just tried this and after I added the 'use_archive_transformativeworks_org:true' to my personal.ini file it seemed to work just fine. THANK YOU!!!!!

Originally Posted by JimmXinu

(Very) Test Version Attached

Nobody said anything about the version I posted that did archive.transformativeworks.org as a separate site.

This version has a switch to force the usual archiveofourown.org adapter to instead use archive.transformativeworks.org for all network requests:

[archiveofourown.org]
use_archive_transformativeworks_org:true

Note that it cannot log in because log in doesn't work on archive.transformativeworks.org.

Other than the actual network reqs, this version still uses and records the normal AO3 domain.

This seems to work for me. I'd like to hear other peoples results.

[kayydotmurr]

Originally Posted by JimmXinu

(Very) Test Version Attached

Nobody said anything about the version I posted that did archive.transformativeworks.org as a separate site.

This version has a switch to force the usual archiveofourown.org adapter to instead use archive.transformativeworks.org for all network requests:

[archiveofourown.org]
use_archive_transformativeworks_org:true

Note that it cannot log in because log in doesn't work on archive.transformativeworks.org.

Other than the actual network reqs, this version still uses and records the normal AO3 domain.

This seems to work for me. I'd like to hear other peoples results.

So to check if this version works, I add two hashtags before my login info in personal.ini right?

[kayydotmurr]

I attempted a few books, I was prompted to log in twice, one got hidden by the author, so that's null, and then I got two 525 errors, and two that updated, which seems like about how well it was going before making these changes.

(these were prompts to update, not fetching new books from URL.)

Unless, of course, I made the plugin change incorrectly which is possible because I'm new to all this.

[booster-t]

I added "use_archive_transformativeworks_org:true" to the ini file and so far (fingers crossed) it has worked with v4.43.7

Stephen

Edit March 16: Well, still not perfect. Almost every story I follow updated except this one and it wants me to log in. When I enter the login information, it still barfs.

[Thorne3]

Originally Posted by JimmXinu

Please don't post NSFW links on Mobile Read. I've removed them for you this time.

The include_images:true directive in defaults.ini is under [epub]. [epub] as a format, takes precedence over both [defaults] and [<site>] sections in personal.ini. There's a wiki page about FFF INI including precedence.

You can either use:

[<fqq domain>:epub]
include_images:false
## for one site, or

## for all downloads.
[overrides]
include_images:false

Thank you for your held and the edit.
I didn't realise there was a wiki and only read the defaults ini file but it might have been a bit too complex for me.

[JimmXinu]

New Test Version Posted

2025-03-16
- adapter_ficbooknet: Fix additional metadata collection, thanks dbhmw
- Add support for 'directimages' with use_browser_cache, thanks nsapa #1180
- AO3 use_archive_transformativeworks_org option