[GUI Plugin] FanFicFare

[jazzhands]

Originally Posted by JimmXinu

The inconsistency of it still argues for a site/server site problem to me.

Is it also happening to people with username/password set in personal.ini?

Yes, it's happening to me with the username/password set in personal.ini.


But I agree that it's related to the current AO3 issues.

Update: I just restarted Calibre and it's now working for the anthology that wouldn't update before.

[swizzlespoon]

I think it has to be something awry with AO3. I hope they're aware of the problem and working on it.

While on the site, I keep having to refresh my browser tab to leave Kudos and sometimes to bookmark or comment. Getting a lot of "session timed out" errors (even though I remain logged in).

And when I'm using FanFicFare (with my login info in personal.ini as always), sometimes I'm prompted for username/password. And it doesn't seem to hold for the whole download session either, I'm prompted for each fic if I have a dozen URLs in my window. Other times (without restarting calibre or anything) it will work as usual.

Just cleared my browser cache again & logged in fresh. And restarted calibre. And so far things are working fine. (But in the past when I did this it would be okay for a bit before the bad behavior started again. Maybe this time all will be well for good!)

[nephtys59]

AO3 is up at the moment. I access through browser with no problems.
Personal.ini has already my username and password memorized.
Updated single fics = no issue.
Try anthology update = error "failed to login".
Are you sure it's not plugin related?

[JimmXinu]

Originally Posted by swizzlespoon

Just cleared my browser cache again & logged in fresh. And restarted calibre. And so far things are working fine. (But in the past when I did this it would be okay for a bit before the bad behavior started again. Maybe this time all will be well for good!)

While acknowledging that 'restart, try again' is a mantra in computing for a reason, I think we're getting into "wave a dead chicken" territory on this one.

Originally Posted by nephtys59

AO3 is up at the moment. I access through browser with no problems.
Personal.ini has already my username and password memorized.
Updated single fics = no issue.
Try anthology update = error "failed to login".
Are you sure it's not plugin related?

Sure? No. But there's enough people having it work and fail intermittently that site/server problem is still my best guess.

Due to the way AO3 implements 'restricted' stories, you may have 'always_login:true' all the time.

Since the errors seem to express as login difficulties, I'd turn off 'always_login:true' any time you don't actually need it.

[fetihif112]

Hello, I have a small problem unrelated to the AO3 issue.

First of all, judging by the monthly release dates, is version 4.38 supposed to be the most recent one on PyPI? Because it isn't, it's still at 4.37.

I'm on Fedora 39, running Calibre 7.3.0, Python 3.12.6. Apparently, the Calibre plugin can use files from the CLI installation when you have both installed. That causes problems when the versions are different - specifically Calibre version 4.38 with CLI version 4.37 breaks downloading from Calibre. Other times in the past everything must have worked fine without me noticing. In this case, I assume it's about the new 4.38 searching feature: all URLs (from various sites, existing in the library or not) fail with error "module 'fanficfare.adapters' has no attribute 'get_url_search'". Uninstalling the CLI version and restarting Calibre fixes this. Reinstalling the CLI version and restarting Calibre breaks it again. Reinstalling 4.38 from within Calibre doesn't help as long as CLI 4.37 exists. I've tried multiple times, and these observations have been consistent.

I've had some trouble diagnosing this, but i'm 95% sure this is it. When all jobs instantly fail, the usual logs in the Jobs window are unavailable, so I had to use debug mode. Even in debug mode, the logs say "Successfully initialized third party plugins: FanFicFare (4, 38, 0)" and "FFF: DEBUG: 2024-09-24 05:58:40,116: calibre_plugins.fanficfare_plugin.fff_plugin(1140) : FanFicFare v4.38.0", with no indication of 4.37's presence other than the errors. I've looked at pip's install/uninstall process, and it doesn't touch any dependencies.

That in and of itself is not a big deal, reinstalling the CLI version whenever I need it takes seconds, but for correctness' sake, how would I go about having Calibre always prioritize its "natiive" version? If it involves error-prone messing with env/path stuff, which I'm unfamiliar with, is there some way of verifying from within Calibre what files it tries to load from where and make troubleshooting easier?

[JimmXinu]

Originally Posted by fetihif112

First of all, judging by the monthly release dates, is version 4.38 supposed to be the most recent one on PyPI? Because it isn't, it's still at 4.37.

That's my mistake. I've uploaded v4.38 to pypi.

Originally Posted by fetihif112

I'm on Fedora 39, running Calibre 7.3.0, Python 3.12.6. Apparently, the Calibre plugin can use files from the CLI installation when you have both installed. ...

That's some pretty good troubleshooting. I agree with your logic--get_url_search was added in FFF v4.38.

Originally Posted by fetihif112

...
That in and of itself is not a big deal, reinstalling the CLI version whenever I need it takes seconds, but for correctness' sake, how would I go about having Calibre always prioritize its "natiive" version? If it involves error-prone messing with env/path stuff, which I'm unfamiliar with, is there some way of verifying from within Calibre what files it tries to load from where and make troubleshooting easier?

I assume you are using Calibre installed from a Fedora repo of some sort. Which has been modified to use python and dependencies installed on the system. The official Calibre download has it's own versions of python and dependencies bundled in precisely because of this sort of issue.

I am reasonably confident installing the official release of Calibre instead would correct this.

Another option would be to not install FFF CLI and install call the FFF plugin as CLI:

calibre-debug --run-plugin FanFicFare -- --help

It's mostly identical to FFF CLI from pypi except for using your personal.ini from either the library specified by new arg --library-path or your last used library (if I recall correctly, it's late). It does not automatically add or update to the library like the GUI plugin does.

[sassejane]

I'm still getting login issues with AO3. Username/password in personal.ini, have always_login, is_adult, and use_view_full_work set to TRUE. After the initial failure I relaunched in Debug and have attached the log in case it's not an AO3 issue.

Sequence:

1. Launched Calibre fresh, hit "Get Story URLs from Email", got prompted for my username/password twice for each of the three stories (cancelled each time)
2. Copied URLs from 'failed to import' dialog, hit "Download from URLs" and pasted the three URLs. Got prompts for each x 2, cancelled each time.
3. Tried Download from URLs again, entered username/password x2 for each story, still failed.
4. Restarted Calibre in Debug mode.
(this is a little hazy but I think this is the order):
5. Hit "Download from URLs" with the three URLs, prompted for password x2, cancelled request, it failed.
6. Hit "Download from URLs", entered username/password x2 for each, failed.
7. Hit "Download from URLs", dismissed password requests, failed.
8. Went back to the Gmail account and marked them unread, hit "Get Story URLs from Email", entered username/password x2 when prompted for each story, failed.
9. Hit "Download from URLs", not prompted for username/password, all three downloaded as expected.

Here's the log:
20240925_fff_debug.txt

[DragoLord19d]

I'm also having AO3 trouble with the windows calibre

[steltek]

Related to the A03 issue, does FFF present a user agent to the website when it logs in and attempts to download content? The fact that FFF will fail to download while the same content can be immediately accessed via a browser made me wonder about this.

[ownedbycats]

plugin-defaults.ini

## User-agent
user_agent:FFF/4.X

[JimmXinu]

Originally Posted by sassejane

9. Hit "Download from URLs", not prompted for username/password, all three downloaded as expected.

Do you have username/password set in personal.ini? What about always_login:true? Because this suggests we MIGHT be asking for login when we don't need to, if you don't have them set.

I am able to see this behavior myself this morning--and it remains inconsistent. Sometimes login works, sometimes it doesn't.

In the same Calibre session, I just saw work-fail-(time reading logs)-work-work on the same restricted story spread across 20 minutes.

As far as I know, this only affects login. And I suspect FFF sees it more often because of always_login:true being set.

[steltek]

Originally Posted by JimmXinu

Do you have username/password set in personal.ini? What about always_login:true? Because this suggests we MIGHT be asking for login when we don't need to, if you don't have them set.

I am able to see this behavior myself this morning--and it remains inconsistent. Sometimes login works, sometimes it doesn't.

In the same Calibre session, I just saw work-fail-(time reading logs)-work-work on the same restricted story spread across 20 minutes.

As far as I know, this only affects login. And I suspect FFF sees it more often because of always_login:true being set.

I even went to far as to try to override the FFF user agent with the current browser user agent string for Firefox in personal.ini to see if that would fix the issue. My thought was that maybe something during the login was setting off the Cloudflare/A03 site security and that it was rejecting the login because it was maybe detecting that the login was coming from a non-browser application. The first time I tried it, it actually worked with links that were just seconds earlier not working. However, after getting my hopes up, it then reverted to form and started failing at login again a few minutes later.

I think you are right: the dead chicken is not just waving but is flapping in a brisk wind. Whatever is going on is a legitimate A03 server login issue (maybe something to do with Cloudflare's security and changes A03 made in their login process) and that there is nothing that can be done to fix it from the user side beyond doing exactly what you recommended - turning off always_login:true unless it is truly needed.

[JimmXinu]

FYI, AO3 is now showing a banner:

We're aware that many users are experiencing frequent forced logouts and errors. Our apologies for the inconvenience! Our volunteers are working on the problem - it isn't necessary to report it to our Support team. For some suggestions that might help reduce how often you are logged out or receive errors, please visit our Known Issues page.

Where is says:

If you are receiving a forced logout error, "session expired" or "auth error", please log back into your account from the Login page rather than using the login form that pops up in the top corner. Once you have logged back in, try opening the page you were viewing again, rather than refreshing the same tab where you received the error. We are aware that this issue has been happening frequently since Sept. 19, 2024, and are continuing to work on a solution.

This is what's causing the AO3 login issues with FFF.

Removing always_login:true or setting it false is about all we can do in FFF.

[kamille3]

Facing this error - invalid literal for int() with base 10: '' when trying to download certain literotica series.

/authors/SirBigBadWolf/works/stories/all
/series/se/493934889
/series/se/494046255

[JimmXinu]

Originally Posted by kamille3

Facing this error - invalid literal for int() with base 10: '' when trying to download certain literotica series.

/authors/SirBigBadWolf/works/stories/all
/series/se/493934889
/series/se/494046255

It looks like that author is broken, or their works aren't linked correctly or something. For all 'series' under that author, either there aren't any chapters listed on the series page, or it flat out says "Error 404 This work is not available". So broken site, basically.

FFF, perhaps optimistically, assumes that any story there's a URL for will list at least one chapter. And I'm not interested in parsing the author page for the list. There are issues.