[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by chicleeblair

I have the latest/test version, and I'm having to re-enter my login details with every adult download from AO3--it seems like multiple times, but that may be older chapters I haven't updated yet--When I started trying to update things, I had the wrong password in my personal.ini, but I fixed that. I'm not using ".gay" or browser_cache. (Although, I have noticed that on the browser, .gay won't log me in.)

If entering it manually works, that suggests the site is working, at least.

If you have a '%' in your password, those need to be escaped in INI files with '%%'

Other than that, I would guess some issue with user/pass being in the wrong section, misspelling, something like that.

[chicleeblair]

No. it's the same as I've always had it:

[archiveofourown.org]
username:xxxxxxxxxxxx
password:Xxxxxxxxnnnn
use_view_full_work:true
#use_archiveofourown_gay:true
#use_browser_cache:true
#use_browser_cache_only:true
#open_pages_in_browser:true
include_titlepage:true
is_adult:true

[JimmXinu]

As I recall, unless you have always_login:true, FFF should only need to login on AO3 for stories that are deliberately 'restricted' by the author (little blue padlock icon).

'Adult' stories are a separate check and passed with is_adult:true , which you have.

Can you provide a debug log? (Restart Calibre in debug under Calibre Preferences dropdown. Attach as text file. Don't obscure story URL in log.)

[JimmXinu]

Looking for some additional opinions.

Someone on github has asked for more chapter range functionality: calibre_plugin: Support chapter ranges properly.

Generalizing a bit, they want FFF to store the chapter range used during download for a book in library and use it again when doing FFF update or overwrite on that book in future.

The stated use case is that they want to split one story by chapter ranges into several books (eg, 1-100, 101-200, 201-) and be able to do overwrite on each book individually -- they assume changes to earlier chapters and inserted chapters are common.

I believe the more common use case would be the older splits would be left alone and only the latest updated. I know it's something we've worked around in past with first_chapter setting. If implemented, I'm currently leaning towards storing it in an optional custom column, not in identifiers:url as suggested. How to handle update by URL would need to be addressed.

In short, is storing chapter range for a book and reusing it something you would find useful?

[ownedbycats]

There have been cases where I download individual chapters from those huge multifandom oneshot collections (with extensive storyURL includes/excludes/replaces), so may be useful when there's more than one.

[Elaine T]

Originally Posted by JimmXinu

In short, is storing chapter range for a book and reusing it something you would find useful?

Maybe? I know I have occasionally wished for such a feature, but it's been a long time.

Asked my kid, who also thinks it may be useful at times, like the other person, who mentioned those huge multi-fandom one shot collections where you only want five of them out of 25.

[MrMental32]

Looking for some assistance if possible. I've recently been bit by the archival bug after noticing favorite fics getting deleted, and I've been using FFF to save what I can.

My issue at the moment is with sofurry. I followed the direction provided on the github wiki and the default .ini file with setting up the personal.ini for the site. For most of the fics, it works great. But for anything in a gallery flagged by the site as 'Adult', the plug-in fails. Did not notice the issue until I was 30 stories deep and noticed I was missing almost 70 chapters across the all the downloads.

The common thing I can find after testing is thus, single flagged chapters are fine, but if I pull from a gallery/folder with mixed content, or all flagged content, anything SFW downloads fine, but anything and everything that is flagged was missing.

Fairly new to Calibre/FFF as a whole, so any help would be of great assistance and appreciated.

Calibre Version 9.14
FFF Version 4.61.0

No special symbols for username or password, aside from the '@' since the site takes an email to login.

.ini sample below

[sofurry.com]

username:XXXXXXXX
password:XXXXXXXX

is_adult:true

use_basic_cache:true

slow_down_sleep_time:1

[JimmXinu]

Originally Posted by MrMental32

Looking for some assistance if possible. I've recently been bit by the archival bug after noticing favorite fics getting deleted, and I've been using FFF to save what I can.

My issue at the moment is with sofurry. ...

I've opened an issue on github to refer to the dev who added that site.

Please post some story URLs (ideally on github) that demonstrate the problem.

[ernand]

I am having a problem with this one fic on FFN when downloading via browser cache. It opens each fic chapter, except the last one.
It downloads all but the last chapter.
When I try to update the fic, I get an error message saying the source only has 24 chapters, but the fic has 25.
The Metadata shows the date of the last updated chapter as the date of chapter 25.

This is the fic: https://www.fanfiction.net/s/1411668...Sun-Blood-Moon

my personal.ini

Spoiler:

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels. See
## https://github.com/JimmXinu/FanFicFare/wiki/INI-File for more
## details.


## windows:
### Chrome:
browser_cache_path:C:\Users\fefue\AppData\Local\Go ogle\Chrome\User Data\Default\Cache\Cache_Data

## Some sites also require the user to confirm they are adult for
## adult content. Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites? You can use strip_chapter_numbers to strip them off. Just
## want to make them all look the same? Strip them off, then add them
## back on with add_chapter_numbers. Don't like the way it strips
## numbers or adds them back? See chapter_title_strip_pattern and
## chapter_title_add_pattern in defaults.ini.
#strip_chapter_numbers:true
#add_chapter_numbers:true

## Some sites/authors/stories use several br tags for scene/section
## breaks. When set replace_xbr_with_hr:X will cause FFF to search
## for X or more consecutive br tags and replace them with br br hr br.
#replace_xbr_with_hr:3

## Some readers don't show horizontal rule (<hr />) tags correctly.
## This replaces them all with a centered '* * *'. (Note centering
## doesn't work on some devices either.)
#replace_hr: false

## Some sites/authors/stories use br tags instead of p tags for
## paragraphs. This feature uses some heuristics to find and replace
## br paragraphs with p tags while preserving scene breaks.
#replace_br_with_p: false

## Some sites/authors/stories (notably AO3/OTW) add empty p tags where
## they aren't intended by the author during document upload and not
## all authors know how/take the time to fix it. This feature removes
## all "empty" <p> tags, ie, those containing only whitespace or <br>
## tags. Literally: <p[^>]*>\s*(<br/>)*\s*</p>
remove_empty_p: true



[epub]
## Include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible. Images work
## in epub format only. To get mobi or other format with images,
## download as epub and use Calibre to convert.
## true by default, uncomment and set false to not include images.
include_images:true
no_image_processing:true

## Quality level to use when converting images to jpg. Range is 0-100,
## reasonable values likely to be in the range 70-95.
jpg_quality: 100

## If set false, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
## true by default, uncomment and set false to not keep summary html.
keep_summary_html:true

## If set true, and there isn't a specific cover image, the first
## image found in the story will be made the cover image. If
## keep_summary_html is true, images in the summary will be before any
## in chapters.
## true by default, uncomment and set false to turn off
make_firstimage_cover:true

## Some attributes cause problems for EBook readers. By default,
## FanFicFare will remove all attributes except the ones specified
## from all tags. (The only exception is that <img> tags will also
## keep src, alt and longdesc attributes. data-orighref is used by
## internalize_text_links to preserve links when chapters are
## inserted.)
## Example: To add 'style', 'title' and 'align' to the list to keep,
## in your personal.ini [defaults] put:
add_to_keep_html_attrs:,style,align
keep_html_attrs:href,name,class,id,colspan,rowspan ,data-orighref


## Most common, I expect will be using this to save username/passwords
## for different sites. Here are a few examples. See defaults.ini
## for the full list.

[www.fanfiction.net]
add_to_output_css:
.storytext {
text-align: left;
}

##use_basic_cache:true
use_nsapa_proxy:true
use_cloudscraper:false

## Errors on more common during long downloads. This way, at least
## the chapters the worked are saved. Often running an update will
## fix the error chapters.
continue_on_chapter_error:true

## browser cache will only be used if use_browser_cache:true and ONLY
## for a few sites. Requires a browser_cache_path set in [defaults].

## use_browser_cache_only:true prevents FFF from falling through to
## request from the server when a page isn't in the browser cache.
## You can turn this on to be sure you're only using the browser
## cache.
use_browser_cache:true
use_browser_cache_only:true
browser_cache_age_limit:4.0
open_pages_in_browser:true
slow_down_sleep_time:2

## fanfiction.net's 'cover' images are really just tiny thumbnails.
## Set this to true to never use them.
#never_make_cover: false

## exclude fanfiction.net imageu (old user vs story detect) and
## default user (d_60_90)
cover_exclusion_regexp/imageu/|d_60_90\.jpg)

## April 2015, ffnet changed their story and user pictures urls to be
## the same. Now the only way to know if the story image is really
## the author image is to go get the author image and check.
skip_author_cover:true

## fanfiction.net is blocking people more aggressively. If you
## download fewer stories less often you can likely get by with
## reducing this sleep.
slow_down_sleep_time:12

## sites are sensitive to too many hits. Users are sensitive to long
## waits during the initial metadata collection in the foreground.
## When used, these settings will speed up metadata downloads in the
## foreground linearly.
tweak_fg_sleep:true
min_fg_sleep:4.0
max_fg_sleep:12.0
max_fg_sleep_at_downloads:4

## Extra metadata that this adapter knows about. See [archiveofourown.org]
## for examples of how to use them.
extra_valid_entries:reviews,favs,follows

## ffnet uses 'Pairings', not 'Relationship', stating they don't have
## to be romantic pairings.
ships_label:Pairings

## Date formats used by FanFicFare. Published and Update don't usually have
## time, but they do now on ffnet.
## See http://docs.python.org/library/datet...ptime-behavior
## Note that ini format requires % to be escaped as %%.
#dateCreated_format:%%Y-%%m-%%d %%H:%%M:%%S
datePublished_format:%%Y-%%m-%%d %%H:%%M:%%S
dateUpdated_format:%%Y-%%m-%%d %%H:%%M:%%S

## ffnet used to have a tendency to send out update notices in email
## before all their servers were showing the update on the first
## chapter. It generates another server request and doesn't seem to
## be needed lately, so now default it to off.
check_next_chapter:false

## ffnet sometimes doesn't update the metadata on the first chapter
## immediately. meta_from_last_chapter:true will instruct the adapter
## to take the metadata from the last chapter instead of chapter 1.
## It can generate another server request and usually isn't necessary,
## so it defaults to off.
meta_from_last_chapter:false

[JimmXinu]

Originally Posted by ernand

I am having a problem with this one fic on FFN when downloading via browser cache. It opens each fic chapter, except the last one.
It downloads all but the last chapter.

This is, unfortunately, common on ffnet, and has been for more than a decade. I used to attribute it to delays in server caching, but more recently, I suspect it's deliberate.

I believe setting either of these 'true' temporarily would work in this case:

[www.fanfiction.net]
## ffnet used to have a tendency to send out update notices in email
## before all their servers were showing the update on the first
## chapter.  It generates another server request and doesn't seem to
## be needed lately, so now default it to off.
check_next_chapter:false

## ffnet sometimes doesn't update the metadata on the first chapter
## immediately.  meta_from_last_chapter:true will instruct the adapter
## to take the metadata from the last chapter instead of chapter 1.
## It can generate another server request and usually isn't necessary,
## so it defaults to off.
meta_from_last_chapter:false

Adding config settings for a single download job is what the INI Snippets feature is for, BTW. I have one saved for just this case:

## Saved as "ffnet check_next_chapter"
[www.fanfiction.net]
check_next_chapter:true
try_shortened_title_urls:true

try_shortened_title_urls is for when the normal chapter URL fails -- we've found taking a letter or two off the story title in the URL will often work. I need to document that setting in defaults.ini.

[Tharos]

Originally Posted by ownedbycats

Has anybody found any sort of pattern to what will trigger AO3 to start giving 525 errors? It seems to be random to me. Even when downloading several stories at once I get the error on just one or two.

It seems (at least for me) that error 525 occurs more often than average when you have to log in to view the story (always_login:true) or when you have to confirm that you are an adult (is_adult:true).

[ernand]

I sometimes get 525 when the fic just updates or the fic has images.
I sometimes get a login token error instead. But if I just keep trying, it will eventually work, or just give it a break of an out for it to work.
It seems to be a server-side issue imo.

[hrbngr]

re: process

for whatever reason, when I click the main fanfarefic icon in calibre, it will only come up w/the Update Existing List option. This happens when i have just downloaded a series or story from literotica, and when i go to do the next one, I can't get the "Download from URLs" box to come up unless I manually choose it from the dropdown on the right of the FFF icon. I tried to disable any sort of "update" setting but this doesn't change the behavior. Any advice on how to bypass this and just come up with the standard "dload from url" option would be great.
thanks.

[hrbngr]

Hello,
I am having trouble getting the fff plugin to work at this time. When i click the icon, i get Update Existing List rather than Download from URLs whenever I have previously downloaded something. I would like to disable this option if possible, but I have not found any way to do this via the Configure Fanficfare options. It does seem to work properly if i delete any existing downloads and then restart Calibre and try again.

[hrbngr]

I am unable to edit my original post, but it appears that if any of the previous d-loaded stories are high-lighted or selected, than it will always try to update that story, even if it was just downloaded, once no stories are selected it will default to "dload from url", and if the url is already copied, it will already be inserted. hope this helps someone, and thanks for this great plug-in to calibre