[GUI Plugin] FanFicFare

[wildfire070]

I got an error while trying to download a story from AO3 today. Never had this error show up before:

'NoneType' object has no attribute 'get_text'

Any ideas?

[JimmXinu]

Nope. Without a story URL, I can't even try the same story.

A debug log would show where the problem occurred. Retry with Calibre in debug (under Calibre Preferences dropdown.)

[FlyingSpaceOxen]

FictionMania now gives "429 Too Many Requests" errors should one exceed a very low threshold of total user activity via web browser and/or FanFicFare, with an accompanying message of "If you're a human being, try again tomorrow." A post by a site admin on the hyberboard states that this was done to combat users mass downloading stories and bots.

I got around this by adding "slow_down_sleep_time:10" under [fictionmania.tv], but care should be taken to avoid clicking any links on the site while FFF is working, otherwise your IP address is liable to be blocked till the next day. The default value for this site should probably be raised accordingly.

The site also seems to instantly block you when it detects FFF's default "user_agent:FFF/4.X". This occurred several times before I changed it to mimic a Firefox user agent.

Additionally, "Story with Images" URLs are not being detected by FFF, with the error: "adapter_fictionmaniatv.py(175): Story With Images(SWI) not found, falling back to HTML."

[JimmXinu]

I have no problem setting default slow_down_sleep_time:10 and clearing the default user_agent for FictionMania.

Originally Posted by FlyingSpaceOxen

Additionally, "Story with Images" URLs are not being detected by FFF, with the error: "adapter_fictionmaniatv.py(175): Story With Images(SWI) not found, falling back to HTML."

I don't read FictionMania myself, so I don't know what that means off hand.

Looking at the adapter, it's not one I wrote, either. And that the site was originally text only with html support wedged in later.

Looking at the site, it's not immediately obvious to me what's different between 'html' and 'SWI'. But I'll look into it tomorrow.

[FlyingSpaceOxen]

Originally Posted by JimmXinu

Looking at the site, it's not immediately obvious to me what's different between 'html' and 'SWI'. But I'll look into it tomorrow.

Here's a random example of a standard HTML story URL:
https://www.fictionmania.tv/stories/...43749251351680

...and a SWI story URL:
https://www.fictionmania.tv/stories/...57645259448893

[Amalthia]

Originally Posted by JimmXinu

First problem I see in the log is that FFF is trying to log in which doesn't work for AO3 through BrowserCache. Comment out username to prevent that.

(There's a whole history about get urls from page logging in even without always_login for reasons.)

You might try use_archive_transformativeworks_org:true (and username commented out) instead of browser cache, that's been working well for me.

This post solved all my browse caching problems! I didn't realize I'd need to comment out my Ao3 login information.

Thanks for creating the cache downloading option, it's been so helpful during Ao3's wait for new servers.

[JimmXinu]

New Test Version Posted

2025-04-09
- adapter_fictionmaniatv: Updates for site change

Originally Posted by FlyingSpaceOxen

Here's a random example of a standard HTML story URL:

That first one is a text story--readtextstory.html. The URLs for 'HTML' and 'SWI' can both be readhtmlstory.html, and older HTML stories can be readxstory.html.

The problem is that the site HTML doesn't have any internal id or class attributes for FFF to find chapter text with. So we were left looking for inline styles to find the chapter text. And that style has changed. For some stories. Inconsistently.

The only consistent marker I see is an HTML comment, so we'll try that. I tested with a handful of stories from now and from 2020. Please report any story URLs that don't work -- NSFW via PM or by posting the storyID number only, not the full URL so-as to not link to NSFW directly.

[steltek]

I've been getting a weird error in trying to download a fic from Spacebattles.

https://forums.spacebattles.com/thre...er-au.1223009/

Whenever I try to download it (I've tried for several days now), it always throws an "None of the 1 URLs/stories given can be/need to be downloaded. See log for details".

In the log, it shows Status: Error, Title: Unknown, Author: Unknown, Comment: "Time Period", and URL: https://forums.spacebattles.com/thre...er-au.1223009/

Every other SB download I do works without any problem, except for this single one. Wondering if it has something to do with the length of the URL or something.

Any ideas?

[JimmXinu]

Originally Posted by steltek

In the log, it shows Status: Error, Title: Unknown, Author: Unknown, Comment: "Time Period", and URL: https://forums.spacebattles.com/thre...er-au.1223009/

Should already be fixed in the current test version.

2025-04-01
- base_xenforoforum: Add timeperiodtags and better handle unexpected typed tags

[steltek]

Originally Posted by JimmXinu

Should already be fixed in the current test version.

2025-04-01
- base_xenforoforum: Add timeperiodtags and better handle unexpected typed tags

You were right - test version resolved the issue.

Thanks!

[jesscat]

Originally Posted by JimmXinu

(Very) Test Version Attached

Nobody said anything about the version I posted that did archive.transformativeworks.org as a separate site.

This version has a switch to force the usual archiveofourown.org adapter to instead use archive.transformativeworks.org for all network requests:

[archiveofourown.org]
use_archive_transformativeworks_org:true

Note that it cannot log in because log in doesn't work on archive.transformativeworks.org.

Other than the actual network reqs, this version still uses and records the normal AO3 domain.

This seems to work for me. I'd like to hear other peoples results.

This works great for me! It does lose my AO3 bookmark status metadata column but that's a small price to pay.

But one thing I don't understand about this site: since there's no way to log in, that seems to mean that stories that require users to be logged in on AO3 just aren't accessible at all on archive.transformativeworks.org, so this fix can't work for them. Or am I missing something?

[JimmXinu]

Originally Posted by jesscat

This works great for me! It does lose my AO3 bookmark status metadata column but that's a small price to pay.

But one thing I don't understand about this site: since there's no way to log in, that seems to mean that stories that require users to be logged in on AO3 just aren't accessible at all on archive.transformativeworks.org, so this fix can't work for them. Or am I missing something?

Correct.

[PandathePanda]

Status
Title
Author
Comment
URL
Error
Not sure why this is happening:

Pokemon: The Origin of Species

HTTP Error in FFF '403 Client Error: Forbidden for url: https://www.fanfiction.net/s/9794740/1/'(403)
https://www.fanfiction.net/s/9794740/1/

Same problem when trying to download a new story:

https://www.fanfiction.net/s/1445875...orld-DC-Comics

[ownedbycats]

You haven't run into that in the last 4 years?????

https://github.com/JimmXinu/FanFicFa...-fanfictionnet

[FlyingSpaceOxen]

Originally Posted by JimmXinu

The problem is that the site HTML doesn't have any internal id or class attributes for FFF to find chapter text with. So we were left looking for inline styles to find the chapter text. And that style has changed. For some stories. Inconsistently.

The only consistent marker I see is an HTML comment, so we'll try that. I tested with a handful of stories from now and from 2020. Please report any story URLs that don't work -- NSFW via PM or by posting the storyID number only, not the full URL so-as to not link to NSFW directly.

Okay, I've tried the test version on random SWI stories ranging from the past 20 years, with differing styles as you've noted. More often than not FFF retrieves them without issue, with the following exceptions.

Occasionally the ad and donation links that are at the bottom of every page on the site are included within the downloaded result; a minor thing but the ads are frequently NSFW regardless of story content. Their readhtmlstory storyID numbers as follows:
33038740531430700263
1429956980574453274
12561299851119015

Then I tried to find SWI stories that consistently failed. These entries' dates place them among the earliest uploaded to the site and seem to have similar formatting.
300645328460827266
30459763732145259
298260336937450