[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by mmmmmmmm

should i just change everything back to the default and go from there?

Honestly, it would probably be easier. From what I see, this is all you have for real setting changes:

Spoiler:

[defaults]
is_adult:true

strip_chapter_numbers:true
add_chapter_numbers:true

custom_columns_settings: 
 fandoms=>#fandom,a

[epub]
make_firstimage_cover:false

[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX

Originally Posted by mmmmmmmm

you mentioned FFF usually calls AO3's Free Form Tags and Categories in genre and Fandoms in category. does that mean i should change the names of my columns?

You can name your custom columns whatever you want. FanFicFare doesn't populate custom columns based on names, only what you configure.

There's two ways to configure FFF to populate a custom column, the Custom Columns tab in FFF Config lists your custom columns and will let you select an FFF standard metadata entry to put in each.

Standard metadata entries are the ones used by all sites/stories.

Site specific metadata entries can only be set to custom columns using custom_columns_settings and you must have the 'Allow custom_columns_settings from personal.ini to override' checkbox enabled (toward bottom of Custom Columns tab).

Unless AO3 is the only site you read, you're probably better served filling your 'Fandoms' column with Category for all sites instead of just site specific fandoms for AO3.

Originally Posted by mmmmmmmm

here is a story ive tried downloading:
https://archiveofourown.org/works/24280306

problems:
- nothing shows up in my fandoms colomn.
- all the tags from all the categories are showing up in my tags column
- nothing shows up in content rating (mature, teen and up, etc. )
- the relationships column sort of works. 'characterA/characterB' tags show up fine, but 'characterA & characterB' tag doesnt really work? like it shows up, but as separate tags? like 'characterA' & 'characterB', so i cant filter by those tags. is there a way to fix that?
- nothing shows up in my category column (M/M, F/F, etc.) but you mentioned FFF calls them genre? should i rename the coulumn? im happy to do that, if it will help

Taken in order:

Fandoms column: Your custom_columns_settings is still broken--need remove the extra colon. It's just => not :=>. Also make sure the 'Allow override' checkbox is checked.

Tags: Unless you change what FFF puts in Tags using the extra_subject_tags setting, that's the default.

Content rating: Not going to be put in any column unless you configure it. FFF labels it just Rating in GUI or lowercase rating for custom_columns_settings. (Calibre's built-in Rating column is completely different--that's a 1-5 stars rating.)

Relationships '/' vs '&' -- As part of each "Comma separated text" custom column definition there's a checkbox for "Contains names". If checked, Calibre automagically splits on '&'.

Category column (M/M, F/F, etc): FFF records that as site specific ao3categories and includes it (and freeformtags) in the standard entry genre.

Saved Metadata Column
FYI, since you are apparently just starting out, I also recommend creating a Long Text type custom column for FanFicFare Metadata and setting FFF to use it (toward bottom of Custom Columns tab).

Then whenever you download or update a story with FFF, it's raw metadata will be saved there and you can use FFF's "Update Calibre Metadata from Saved Metadata Column" mode to update existing books as your library configuration evolves without hitting the servers again.

[Homer55]

Hi!, hope someone can help (if it's even possible).

is there a way to make sure fanficfare only checks for updates after certain chapters? I found some stories that get separated by books each with it's own cover and stuff but the problem is that everything is in the same thread, and what I want to do is separate each book into it's own epub and make sure each book has it's own cover and stuff and leave those alone and only the last book would get updated everytime there's a new chapter but the problem is, if I delete all previous chapters, fanficfare detects that there are less chapters than the total so it downloads them and I just want it to download the new ones. Is there a way to tell it, ignore everything from chapter # and back? and only inject the new ones at the end?

[JimmXinu]

You can set first_chapter:50 (or whatever) under story URL section in personal.ini and FFF will treat it as if you'd used <storyURL>[50-] every time.

This is technically 'undocumented', but it works.

There's also an ignore_chapter_url_list feature.

[John Maverick]

Originally Posted by JimmXinu

Assuming you are using the default port numbers and running both on the same computer, I would suspect that there's a firewall module or something blocking the connection.

I checked the firewall of my PC but didn't find any different. Also tried reinstalling the Docker container of FlareSolverr, but is still giving me the same result.

Also visiting localhost:8191 in the browser shows FlareSolverr running.

I'll try to see what more could be wrong in my setup.

As always thanks for your help

[Homer55]

Originally Posted by JimmXinu

You can set first_chapter:50 (or whatever) under story URL section in personal.ini and FFF will treat it as if you'd used <storyURL>[50-] every time.

This is technically 'undocumented', but it works.

There's also an ignore_chapter_url_list feature.

This worked wonders! thank you so much!

[DaPoke]

Is there a way to rename tags for websites that tend to use abbrievated versions. Like for example rename a tag "abc" => "an actual tag name"

[JimmXinu]

Yes, you can search this thread for replace_metadata, there's also a wiki page ReplaceMetadata.

[DaPoke]

Originally Posted by JimmXinu

Yes, you can search this thread for replace_metadata, there's also a wiki page ReplaceMetadata.

Ah perfect thank you! Putting my answer here for others to see!

I had to actually not to check the genre,category metakeys, but the extratags

Something like this:

replace_metadata:
extratags=>\bsf\b=>science fiction

I had to include the regex to match whole words using `\b` otherwise it would process the next ones and tags would be a weird screwed up mix of replacements.

[mmmmmmmm]

Originally Posted by JimmXinu

Content rating: Not going to be put in any column unless you configure it. FFF labels it just Rating in GUI or lowercase rating for custom_columns_settings. (Calibre's built-in Rating column is completely different--that's a 1-5 stars rating.)

Category column (M/M, F/F, etc): FFF records that as site specific ao3categories and includes it (and freeformtags) in the standard entry genre.

Saved Metadata Column
FYI, since you are apparently just starting out, I also recommend creating a Long Text type custom column for FanFicFare Metadata and setting FFF to use it (toward bottom of Custom Columns tab).

thank you! i've been able to fix most of them now!

im still struggling with Content rating and Category/rating.

i went on to the 'custom columns' section and set that column to 'rating' from the drop down menu. that didnt fix it

i also tried fixing them in the custom_columns_settings:

Spoiler:

custom_columns_settings: 
 fandoms =>#fandom,a
 genre =>#genre,a
 rating =>#rating,a

thanks for letting me know about the Saved Metadata Column, that sounds super helpful and i've set it up now!

[JimmXinu]

I'd double check that you have the custom column identifiers correct for your created columns (#genre vs #genres etc) and that column types are appropriate. You can't put text/tags in an integer field, for example.

[John Maverick]

Originally Posted by JimmXinu

Assuming you are using the default port numbers and running both on the same computer, I would suspect that there's a firewall module or something blocking the connection.

I tested using FlareSolverr from the cli with curl, to see if this is a firewall problem or anything else. It can get a susesful response with webs that don't have Cloudflare challenges, like google.com or php.net.

But when trying to do the same with a story of ff.net like this:

curl -L -X POST 'http://localhost:8191/v1' \
-H 'Content-Type: application/json' \
--data-raw '{
  "cmd": "request.get",
  "url":"https://www.fanfiction.net/s/14321954/1/Dragon-Ball-M-Mirai-New-Future",
  "maxTimeout": 120000
}'

I get this response:

{"status": "error", "message": "Error: Error solving the challenge. Timeout after 120.0 seconds.", "startTimestamp": 1706570463731, "endTimestamp": 1706570584578, "version": "3.3.13"}

It seems more like for some reason FlareSolverr can't resolve the challenge. But is strange that this seems only happening to me. Anyone else is getting similar errors?

[JimmXinu]

Depending on your download volume and habits, and possibly your network neighbors' habits, your IP or network may be on a list for more aggressive blocking.

I use Browser Cache instead of Flaresolverr because I find it more reliable, if less convenient.

I might look at improving the error reporting, however. I thought FFF reported back any error message it got from Flaresolverr.

UPDATE:

Actually, it looks like Flaresolverr doesn't work for me either with that story URL. The arms race continues.

[John Maverick]

Originally Posted by JimmXinu

I might look at improving the error reporting, however. I thought FFF reported back any error message it got from Flaresolverr.

I always through that that was the expected error message, maybe because some limitation. Good to know that it'll be possible to see those error from Calibre.

Originally Posted by JimmXinu

Actually, it looks like Flaresolverr doesn't work for me either with that story URL. The arms race continues.

Thanks for checking this. Now at least I know that it isn't something weird on my end.

[ownedbycats]

A recent ticket on the Flaresolverr github mentions a challenge loop. Interestingly, the ticket is specifically mentions FanFiction.net and I believe the reporter is also active on the FFF github.

[JimmXinu]

New Test Version Posted

2024-01-29
- fetcher_flaresolverr: Report errors from Flaresolverr better and fail faster--no retries.

Won't fix anything, but should make it somewhat easier to tell what's going on.