[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by applegaa

The test version was working great and I didn't run into any issues with it.
I finally got around to installing the new monthly version today and it doesn't appear to have this functionality. Was there a reason it got pulled?

- Andrew

That change is included in the released version. I double checked the posted zip.

[applegaa]

Originally Posted by JimmXinu

That change is included in the released version. I double checked the posted zip.

Calibre shows version 4.37.0 as Installed and Available.

I just restarted in debug mode and then used the following as my test cases. I did verify that they all ended up at the first URL when entered in my browser.

https://www.scribblehub.com/series/1...rawler-litrpg/
https://www.scribblehub.com/series/1...ake-name-here/
https://www.scribblehub.com/series/1...fake-name-here
https://www.scribblehub.com/series/1144593/
https://www.scribblehub.com/series/1144593

The "fake-name-here" URLs actually matched the existing story and asked if I wanted to update the URL. I'm not sure why it worked today but not last night. I'll keep watching and see if it happens again.

However, the ability to enter a URL without a title still doesn't work. The last two URLs above were just ignored by FanFicFare. It wouldn't pick them up from the clipboard and did nothing if I manually pasted them in. That's not a critical feature for me, but I wanted to mention it.

I'm not including the debug log because the title change worked and the no title URLs were not even logged.


For the record, from last night's failure, this was the old URL:
https://www.scribblehub.com/series/8...-thief-litrpg/
And this was the URL that resulted in a new epub:
https://www.scribblehub.com/series/8...-arcane-thief/
The author's name and URL did not change. Also, yes, this is the same story that prompted my original request. Snollygoster really likes to change the name around.

Okay, new wrinkle... Before posting this I got the bright idea, since it was working now, to delete the new epub and update the old one. I didn't think to restart in debug mode this time, sorry, but I gave it the new URL and it's now downloading all 193 chapters again instead of updating the existing 192 chapter epub.

After using the "new" URL from last night, I discovered that the name and URL has changed again. The newest new one is:
https://www.scribblehub.com/series/8...-thief-litrpg/

Curious what would happen, I dumped that URL into FanFicFare. Rather than complaining that it couldn't determine which book to update or simply downloading a third copy of it, it actually matched up to the new epub and asked if I wanted to update the URL. However, it didn't actually update it. All that it updated was the Title itself. I'll try again when there is actually a new chapter to add, as maybe the URL change got dropped without that update?

I am so confused right now...

- Andrew

[JimmXinu]

I've commented before that I don't read that site. So when I make a change to address a reported issue, that's generally the only thing I'm looking at--I'm depending on users to report any related issues or knock-on effects.

I will look at adapting the logic we use for ffnet to scribblehub.

Fair warning, I expect that some form of title will be necessary. These probably won't work:
https://www.scribblehub.com/series/1144593/
https://www.scribblehub.com/series/1144593

[riothamus]

Originally Posted by applegaa

Curious what would happen, I dumped that URL into FanFicFare. Rather than complaining that it couldn't determine which book to update or simply downloading a third copy of it, it actually matched up to the new epub and asked if I wanted to update the URL. However, it didn't actually update it. All that it updated was the Title itself. I'll try again when there is actually a new chapter to add, as maybe the URL change got dropped without that update?

I am so confused right now...

- Andrew

Just curious, I've been trying to download from scribblehub the last few days with flaresolverr (even updated flaresolverr) and I'm getting timeouts for detected challenges. Are you having any issues?

[JimmXinu]

New Test Version Posted

2024-08-14
- adapter_scribblehubcom: Allow for changing title in story URL.
- Update adapter_syosetucom.py (#1095) Fix update retrieval for series not completed, thanks dado330
- Make CLI username prompt more visible

I've adapted similar logic to ffnet for scribblehub for INI sections, plugin URL search, and chapter URL normalization.

[applegaa]

Originally Posted by riothamus

Just curious, I've been trying to download from scribblehub the last few days with flaresolverr (even updated flaresolverr) and I'm getting timeouts for detected challenges. Are you having any issues?

I do occasionally, but not that often and I just retry the failed one. Except when the site goes into its quasi-lockdown state, I've only had one ever failed twice in a row.

That said, I don't use flaresolverr, as I never got it to work reliably. Instead, I just added "use_cloudscraper:true" to the "[www.scribblehub.com]" section of my personal.ini. It's not perfect, but it greatly reduced the 403 rate I was seeing before using it.

- Andrew

[riothamus]

Originally Posted by applegaa

I do occasionally, but not that often and I just retry the failed one. Except when the site goes into its quasi-lockdown state, I've only had one ever failed twice in a row.

That said, I don't use flaresolverr, as I never got it to work reliably. Instead, I just added "use_cloudscraper:true" to the "[www.scribblehub.com]" section of my personal.ini. It's not perfect, but it greatly reduced the 403 rate I was seeing before using it.

- Andrew

Thanks for the tip. That worked. Took two tries but it worked. I already had that line commented out in my .ini so I must have tried it in the past. Hopefully they'll fix flaresolverr soonish. Appreciate it.

[applegaa]

Originally Posted by JimmXinu

I've commented before that I don't read that site. So when I make a change to address a reported issue, that's generally the only thing I'm looking at--I'm depending on users to report any related issues or knock-on effects.

I will look at adapting the logic we use for ffnet to scribblehub.

Fair warning, I expect that some form of title will be necessary. These probably won't work:
https://www.scribblehub.com/series/1144593/
https://www.scribblehub.com/series/1144593

I understand you don't use the site and really appreciate that you continue to support it anyways.

I may have misunderstood the wording when you released the original test version. I thought it said that it would accept URLs without titles. It's not a feature I see myself using. I just tested it because I thought it was supported.


It looks like the issue was actually with the metadata for the older epub. It looked fine when I viewed it, but FanFicFare behaved weirdly when trying to update the story directly instead of from the URL. So, I just deleted it and will stick with the new one.

Sorry for not doing more thorough testing before reporting the issue.

- Andrew

[Tharos]

After updating a fanfiction, I also have FFF create a new cover.
Can I set it somehow / somewhere that in certain cases no cover is created? For example, if a certain tag is in the metadata? I use a user-defined column (#cover) in which I have stored tags.

[JimmXinu]

Originally Posted by Tharos

After updating a fanfiction, I also have FFF create a new cover.
Can I set it somehow / somewhere that in certain cases no cover is created? For example, if a certain tag is in the metadata? I use a user-defined column (#cover) in which I have stored tags.

Maybe? Using the Generate Cover plugin it should be possible.

The generate_cover_settings INI setting can be used to change which Generate Cover plugin setting gets used and, assuming you have checked the 'Pass Calibre Columns into FanFicFare' option (personal.ini tab of FFF config), you should be able to use calibre_cust_cover as a metadata entry.

## If you have the Generate Cover plugin installed, you can use the
## generate_cover_settings parameter to intelligently decide which GC
## setting to run.  There are three parts 1) a template of which
## metadata part(s) to look at, 2) a regular expression to match the
## template, and 3) the name of the GC setting to use, which must
## match exactly.  Use this parameter in [defaults], or by site eg,
## [ficwad.com]
## Make sure to keep at least one space at the start of each line and
## to escape % to %%, if used.
## template => regexp to match => GC Setting to use.
## To use this, make sure you go to the Generate Cover tab in FanFicFare
## config and check 'Allow generate_cover_settings from personal.ini
## to override'
## You can check for an existing cover image in the metadata
## cover_image.  cover_image can be:
## specific -- The site has an image that is specifically this story's cover
## first -- The first image in the story or story desc is used as cover
## default -- A default_cover_image was used
## force -- A force_cover_image was used
## old -- There was already a cover image in the epub.
## This is useful because GC plugin can use the existing image.
#generate_cover_settings:
# ${category} => Buffy:? [tT]he Vampire Slayer => BuffyCover
# ${category} => Star Trek => StarTrekCover
# ${cover_image} => ^$ => CompleteCover
# ${cover_image} => (specific|first|default|force) => CoverWithOrigImage

Make sure you also see the cover settings wiki page.

[citac]

How does one reset the personal.ini? Do I just copy over the default.ini? I've been trying various things from various people, and while I like some things there are others that don't give me the results I'd like. I need to start over to try and understand more what these settings are, what falls under genre, what under category, whether adding #ao3_ should be done in lookup names, etc.

I am also wondering, if I decide to create two different libraries (say, a Fiction library and an AO3 library) do changes in personal.ini apply to both or only to the library that was active when I was adding the changes?

[JimmXinu]

Originally Posted by citac

How does one reset the personal.ini? Do I just copy over the default.ini? I've been trying various things from various people, and while I like some things there are others that don't give me the results I'd like. I need to start over to try and understand more what these settings are, what falls under genre, what under category, whether adding #ao3_ should be done in lookup names, etc.

If you delete everything in personal.ini, the default example will be copied in. I do not recommend copying defaults.ini.

Originally Posted by citac

I am also wondering, if I decide to create two different libraries (say, a Fiction library and an AO3 library) do changes in personal.ini apply to both or only to the library that was active when I was adding the changes?

Each library has it's own FFF settings. Changes in one do not affect any other libraries.

For example, I have a copy of libraries used just for testing different FFF settings.

[Wolgan]

I use a lot of email subscriptions to keep up to date on stories, and I've noticed that when I use the Get Story URLs from Email option, it seems to take significantly longer to parse RoyalRoad notifications than any other. It'll speed through 10-15 emails in as many seconds, then sit at a single Royal Road email notification for upwards of 30 seconds.

I had let my notifications build up for quite a while, and had a few hundred for it to go through, and watching as messages ticked over to Read in my mailbox as FFF scanned through them it was very consistent (and fascinating, in a nerdish sort of way). 1-2 seconds per email (at most), until it hits an RR email, then it would hang for a relatively long time.

I'm happy to pack some of the emails up and/or forward to you if it'll help.

Potentially useful info:

• Using dedicated gmail mailbox for notifications.
• Incoming stories are tagged with the "Story Updates" label by gmail rules on receipt.
• Other sites that I get notification for that do not have this issue include: StoriesOnline (via feedrabbit.com rss-to-email service), AO3, fanfiction.net, and hentaifoundry.

Screenshot of my FFF email settings

This is minor, as I usually just work on other stuff while it pulls emails, so low priority/impact (for me anyway).

edit: I dug into the email and I see RR is using custom links that have to be translated/resolved to actual story urls. That would explain the delay and I can't think of any obvious way to get around it. I think I'll start shuffling those off to a different folder/label and manually calling for updates directly from the existing stories in Calibre. I don't have too many of those anyway.

[citac]

Originally Posted by JimmXinu

If you delete everything in personal.ini, the default example will be copied in. I do not recommend copying defaults.ini.

Each library has it's own FFF settings. Changes in one do not affect any other libraries.

For example, I have a copy of libraries used just for testing different FFF settings.

Oh, that's good to know about both the resetting the personal.ini and different settings for libraries, it makes me feel better about experimenting.

I think I need to go back to the metadata management for fanfic to try and work out what exactly I need to see in my library and what custom columns I need to create.

In preparation, I'm hoping that someone might know how to deal with a situation where an author has several series arcs that all follow each other. When sorting by series, they are not sorted in the arc order, but by series. I would like to see my collection sorted by fandom, author and series, and for the series arcs to be sorted correctly. I don't think that's what the multiseries is for - that's more for one fic belonging to several series, correct? How do I achieve what I need?

[JimmXinu]

Originally Posted by Wolgan

edit: I dug into the email and I see RR is using custom links that have to be translated/resolved to actual story urls. That would explain the delay and I can't think of any obvious way to get around it. I think I'll start shuffling those off to a different folder/label and manually calling for updates directly from the existing stories in Calibre. I don't have too many of those anyway.

Yeah, that's special code for RR. I'm a bit surprised I put that in. Somebody may have contributed it--or nagged especially hard for it.