[GUI Plugin] FanFicFare

[John Maverick]

Originally Posted by JimmXinu

I might look at improving the error reporting, however. I thought FFF reported back any error message it got from Flaresolverr.

I always through that that was the expected error message, maybe because some limitation. Good to know that it'll be possible to see those error from Calibre.

Originally Posted by JimmXinu

Actually, it looks like Flaresolverr doesn't work for me either with that story URL. The arms race continues.

Thanks for checking this. Now at least I know that it isn't something weird on my end.

[ownedbycats]

A recent ticket on the Flaresolverr github mentions a challenge loop. Interestingly, the ticket is specifically mentions FanFiction.net and I believe the reporter is also active on the FFF github.

[JimmXinu]

New Test Version Posted

2024-01-29
- fetcher_flaresolverr: Report errors from Flaresolverr better and fail faster--no retries.

Won't fix anything, but should make it somewhat easier to tell what's going on.

[John Maverick]

Originally Posted by JimmXinu

New Test Version Posted
2024-01-29
- fetcher_flaresolverr: Report errors from Flaresolverr better and fail faster--no retries.

I installed the new test version of FFF and tried to download an story from ff.net that go. But I'm still seeing in the same error message

Error
Unknown
Unknown
Connection to flaresolverr proxy server failed.  Is flaresolverr started?
https://www.fanfiction.net/s/14322332/1/Life-s-Weight

[JimmXinu]

Can you run in debug (under Calibre's Preferences dropdown), try again, and post the debug log?

[JimmXinu]

New Test Version

2024-01-30
- adapter_deviantartcom: Changed to 2 post login (#1035) and finding story id

[John Maverick]

Originally Posted by JimmXinu

Can you run in debug (under Calibre's Preferences dropdown), try again, and post the debug log?

I attached the log file in this post. The FFF version I have installed is v4.30.9

[JimmXinu]

I think I know what's happening here now. Please try adding this setting:

flaresolverr_proxy_timeout:59000

Or a smaller value--it's milliseconds.

FYI, all I'm trying to do here is get a better error message.

[John Maverick]

Originally Posted by JimmXinu

I think I know what's happening here now. Please try adding this setting:

flaresolverr_proxy_timeout:59000

Or a smaller value--it's milliseconds.

FYI, all I'm trying to do here is get a better error message.

With 59 seconds the error message seems different, giving an 500 error that wasn't previously.

With 30 seconds did the same that with 59.

Attached both logs and also the secure personal.ini

[JimmXinu]

Yep. That's what I wanted you to see:

"Flaresolverr says: Error: Error solving the challenge"

instead of:

"Connection to flaresolverr proxy server failed. Is flaresolverr started?"

I'll include that change to the default setting in the next version.

[ownedbycats]

On AO3, when an author inserts a new fic mid-series, is the best way to update the numbering from them all to use 'get story URLs" from the series page and then run an update on them all?

[mmmmmmmm]

hi again! im sorry, i just cant seem to figure this out.

i raelly want to be able to filter by M/M or F/F or F/M etc.
i have a column for that, and i know you said FFF stores that in genre (for ao3).

i have no idea what i'm missing, but nothing shows up in my column.

in 'custom columns' i have it set to 'genre'

ive tried to play around with the personal.ini

Spoiler:

## This is an example of what your personal configuration might look
## like.  Uncomment options by removing the '#' in front of them.

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels.  See
## https://github.com/JimmXinu/FanFicFare/wiki/INI-File for more
## details.
include_in_genre:genre,forumtags,freeformtags, ao3categories
extra_valid_entries:fandoms

 category=> (- All Media Types|- Fandom|\(Movies\)|\(Movie [0-9]+\)|\(Comics\)|\(TV\))$=>
 characters=>\(mentions of\) =>
 characters=> \([Ch]haracter\)=>
 characters=> ?\((cameo|briefly|if you squint|sort of)\)=>
 characters=> - Character=>
 characters=>^Other mentions$=>

 ships=>(.*)[/&](.*) \(?[Ff]riendship(?: only)\)?=>\1 & \2
 ships=> - Relationship=>
add_to_replace_metadata:
## Changes line breaks to paragraphs
 description=><br/>=></p><p>
## Applies justified alignment to paragraphs
 description=><p>=><p align="justify">



## The default value is: 
## include_subject_tags: extratags, genre, category, characters, ships, status

## remove all from tags:
include_subject_tags: extratags, genre, 

## Some sites also require the user to confirm they are adult for
## adult content.  Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites?  You can use strip_chapter_numbers to strip them off.  Just
## want to make them all look the same?  Strip them off, then add them
## back on with add_chapter_numbers.  Don't like the way it strips
## numbers or adds them back?  See chapter_title_strip_pattern and
## chapter_title_add_pattern in defaults.ini.
## strip_chapter_numbers:true
## add_chapter_numbers:true


[epub]
## Include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible.  Images work
## in epub format only.  To get mobi or other format with images,
## download as epub and use Calibre to convert.
## true by default, uncomment and set false to not include images.
include_images:true

## If set false, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
## true by default, uncomment and set false to not keep summary html.
#keep_summary_html:true

## If set true, and there isn't a specific cover image, the first
## image found in the story will be made the cover image.  If
## keep_summary_html is true, images in the summary will be before any
## in chapters.
## true by default, uncomment and set false to turn off
make_firstimage_cover:false 


## Most common, I expect will be using this to save username/passwords
## for different sites.  Here are a few examples.  See defaults.ini
## for the full list.


[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true

chapterslashtotal_label: Chapters
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,chapterslashtotal,numWords,publisher,description
add_to_replace_metadata:
 chapterslashtotal=>^(\d+)/\1$=>\1
 chapterslashtotal=>/\?$=>

## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
collect_series: true

extra_valid_entries:fandoms, freeformtags, freefromtags,  ao3categories, comments, chapterslashtotal, chapterstotal, kudos,  hits, bookmarks, collections, byline, bookmarked, bookmarktags,  bookmarksummary, bookmarkprivate, bookmarkrec, restricted, series00,  series01, series02, series03, series04, series05, series06, series07,  series08, series09, series00Url, series01Url, series02Url, series03Url,   series04Url, series05Url, series06Url, series07Url, series08Url, series90Url,  series00HTML, series01HTML, series02HTML, series03HTML, series04HTML,  series05HTML, series06HTML, series07HTML, series08HTML, series09HTML 
fandoms_label:fandoms 
freeformtags_label:Freeform Tags 
freefromtags_label:Freeform Tags 
ao3categories_label:AO3 Categories 
comments_label:Comments 
chapterslashtotal_label:Chapters/Total Chapters 
chapterstotal_label:Total Chapters 
## kudos_label:Kudos 
## hits_label:Hits 
## collections_label:Collections 
## Count of bookmarks on story by all users bookmarks_label:Bookmarks 
## Tags & Summary from *your* bookmark on the story.  Only collected 
## if always_login:true bookmarked_label:I Bookmarked Story bookmarktags_label:My Bookmark Tags 
## bookmarksummary_label:My Bookmark Summary 
## bookmarkprivate_label:My Bookmark Private 
## bookmarkrec_label:My Bookmark Rec 
## restricted_label:Restricted to Registered Users 

custom_columns_settings: 
 fandoms =>#fandom,a
 genre =>#ao3categories,a
 rating =>#rating_label,a
## Question: what does the (,a) after '#fandom' do?

[culytera]

Originally Posted by mmmmmmmm

hi again! im sorry, i just cant seem to figure this out.

i raelly want to be able to filter by M/M or F/F or F/M etc.
i have a column for that, and i know you said FFF stores that in genre (for ao3).

i have no idea what i'm missing, but nothing shows up in my column.

in 'custom columns' i have it set to 'genre'

ive tried to play around with the personal.ini

[defaults]
include_in_genre:genre,forumtags,freeformtags, ao3categories

Including all those metadata entries in genre means you're not only adding ao3categories (F/F, etc.) in your ao3categories column, you're also adding the entries under genre, forumtags (not sure if Ao3 uses that) AND freeformtags. If you only want the F/F entries in that column you just need ao3categories=>#ao3categories under your custom_columns_settings instead of genre =>#ao3categories.

 category=> (- All Media Types|- Fandom|\(Movies\)|\(Movie [0-9]+\)|\(Comics\)|\(TV\))$=>
 characters=>\(mentions of\) =>
 characters=> \([Ch]haracter\)=>
 characters=> ?\((cameo|briefly|if you squint|sort of)\)=>
 characters=> - Character=>
 characters=>^Other mentions$=>

 ships=>(.*)[/&](.*) \(?[Ff]riendship(?: only)\)?=>\1 & \2
 ships=> - Relationship=>

These lines need to be below add_to_replace_metadata.

[defaults]
## Some sites also require the user to confirm they are adult for
## adult content.  Uncomment by removing '#' in front of is_adult.
is_adult:true

[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true

You have redundant is_adult entries.

## Question: what does the (,a) after '#fandom' do?

You can check the default personal.ini to learn what some settings do.

There are three parts, the entry name,
## then the label of the calibre custom column, then (optionally) a
## 'mode'. 'r' to Replace any existing values, 'a' to Add to existing
## value (use with tag-like columns), and 'n' for setting on New books
## only. (Default is 'r'.)

I did a test with the ini below and attached screenshot is the result.

Spoiler:

[defaults]
#include_in_genre:genre,forumtags,freeformtags, ao3categories
include_in_genre:

#extra_valid_entries:fandoms

add_to_replace_metadata:
 category=> (- All Media Types|- Fandom|\(Movies\)|\(Movie [0-9]+\)|\(Comics\)|\(TV\))$=>
 characters=>\(mentions of\) =>
 characters=> \([Ch]haracter\)=>
 characters=> ?\((cameo|briefly|if you squint|sort of)\)=>
 characters=> - Character=>
 characters=>^Other mentions$=>
 ships=>(.*)[/&](.*) \(?[Ff]riendship(?: only)\)?=>\1 & \2
 ships=> - Relationship=>
## Changes line breaks to paragraphs
 description=><br/>=></p><p>
## Applies justified alignment to paragraphs
 description=><p>=><p align="justify">

include_subject_tags:ships,category
is_adult:true


[epub]
include_images:true
make_firstimage_cover:false 

[archiveofourown.org]
extra_subject_tags:ao3categories
chapterslashtotal_label: Chapters
titlepage_entries: seriesHTML,category,ao3categories,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,freeformtags,chapterslashtotal,numWords,description

add_to_replace_metadata:
 chapterslashtotal=>^(\d+)/\1$=>\1
 chapterslashtotal=>/\?$=>

custom_columns_settings:
 ao3categories =>#ao3categories
 rating =>#audience

[JimmXinu]

Originally Posted by ownedbycats

On AO3, when an author inserts a new fic mid-series, is the best way to update the numbering from them all to use 'get story URLs" from the series page and then run an update on them all?

Basically, yes. Or select the books in your library and 'update always'.

It's the same as if they'd changed any other metadata like the title or author.

If you don't care if the title page is updated, Manage Series plugin is useful to manually change series in Calibre only.

To update title pages in an existing anthology, you'd also need to do 'update always', FYI.

[JimmXinu]

New Version Released

Version 4.31.0 - 2024-02-01
- Update translations.
- adapter_deviantartcom: Changed to 2 post login (#1035) and finding story id
- fetcher_flaresolverr: Report errors from Flaresolverr better and fail faster--no retries.
- Update adapter_royalroadcom to remove 'stolen' injections. Thanks, grenskul
- Fix for site change, adapter_deviantartcom. #1030
- Allow image/comic (/i/) and poems (/p/) on literotica
- Fix estimatedWords site metadata for SB/SV - thanks mx-sylveon!
- Swap out SuperFastHash implementation #1026
- Another OTW(AO3) block/hidden story string