[GUI Plugin] FanFicFare

[ackomb]

Hi Guys,

I was just wondering, and I'm not trying to jinx anything, but has anyone had the cloudscraper issue lately?

I've been collection metadata from the website for a bunch of my older stories from when I had no clue how to use the .ini to make structural changes for me (at least now I have some basic knowledge) and I've been following the advise to use the saved metadata column to reduce traffic. But that meant I have to collect it first.
Since there are quite a few stories that were missing the saved metadata, I have been trying to figure out at which number of stories it triggers the error, and how much time to leave in between, but surprisingly since the latest release update of FFF, I installen earlier this week, I've had no issues.
(And no, I have not opened the stories first to use the browser cache for these, which I did set up for new stories.)

And I was just wondering if its just me or if ff.net has finally fixed the whole thing.

When I open all new stories I want to download I've been getting a cloudscraper intermediate screen which takes a few seconds and then it opens without the extra addition of the, how do you call the selecting photo's thingy? so it seems that the longer/variable sleeptime has solved the issue.

Like I said just wondering, and maybe the information is helpful is some form.

[JimmXinu]

It's extremely variable. Just had somebody asking about it over night on the mailing list.

[ownedbycats]

I suspect that FFnet might have different security levels for specific countries/IP ranges (or misconfigured something), which is why some people are having more trouble with it.

[ownedbycats]

Question: In the update log, "Same Story Already Included" results in both title and author being Unknown. It doesnt affect things at all, but is it intended?

[JimmXinu]

Default values that are never updated because the metadata isn't fetched.

[bigwoof]

just checked out the latest 4.0.2 version..

Thank you @JimmXinu for a fantastic release. You work is *much* appreciated!! :-)

[darth1]

Moderator Notice
Removed NSFW links -- JimmXinu

Spoiler:

calibre, version 3.14.0
ERROR: Error Updating Metadata: <p>An error has occurred while FanFicFare was updating calibre's metadata for (link removed).</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1780, in update_books_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 2418, in update_metadata
File "site-packages\calibre\db\legacy.py", line 535, in format
File "site-packages\calibre\db\cache.py", line 893, in format
File "site-packages\calibre\db\cache.py", line 63, in call_func_with_lock
File "site-packages\calibre\db\cache.py", line 743, in copy_format_to
File "site-packages\calibre\db\backend.py", line 1454, in copy_format_to
File "shutil.py", line 52, in copyfileobj
File "tempfile.py", line 626, in write
MemoryError: out of memory

Story Details:status:
Add
reportorder:
0
collision:
Add New Book
title:
Tails of the Reluctant Teitoku
password:
(was set, removed for security)
url:
(link removed)
begin:
None
listorder:
0
is_adult:
False
timestamp:
2021-02-28 19:36:54.658000+13:00
calibre_id:
43305
comments:
<div class="messageText SelectQuoteContainer ugc baseHtml"> So, I've finally gotten around to starting a story of someone who normally would never get to be a harem protagonist. This is the story of him getting dragged kicking and screaming into the craziness that is a Teitoku of ship-girls. <br/> <br/>
A couple of things to remind readers of. In this Kancolle-verse, all ship-girls of destroyer class and above are physically mature and are physiologically and mentally eighteen years of age and/o</div>
updatedate:
2021-02-26 10:08:00+13:00
publisher:
forum.questionablequesting.com
uniqueurl:
(link removed)
pubdate:
2019-07-07 13:44:00+12:00
author:
[u'MagisterAurelius']
added:
True
series:

showerror:
True
tags:
[u'Tentacles Gotta Tentacle', u'Ruined Orgasm', u'Bukkake', u'Kantai Collection', u'Lewd And Smut Ahead', u'Impregnation/Breeding Will Happen', u'NSFW', u'FanFiction', u'Romance', u'Azur Lane', u'Lactation']
good:
True
icon:
plus.png
calibre_columns:

author_sort:
[u'MagisterAurelius']
comment:
Download epub completed, 114 chapters.
outfile:
C:\Users\DARTHC~1\AppData\Local\Temp\calibre_yd84g c\fanficfare_qmlvd6\Tails of the Reluctant Teitoku-MagisterAurelius-rx_akm.epub
all_metadata:
dateUpdated:
2021-02-26 10:08
ships:

output_css:

body { background-color: #ffffff;
text-align: justify;
margin: 2%;
adobe-hyphenate: none; }
pre { font-size: x-small; }
h1 { text-align: center; }
h2 { text-align: center; }
h3 { text-align: center; }
h4 { text-align: center; }
h5 { text-align: center; }
h6 { text-align: center; }
.CI {
text-align:center;
margin-top:0px;
margin-bottom:0px;
padding:0px;
}
.center {text-align: center;}
.cover {text-align: center;}
.full {width: 100%; }
.quarter {width: 25%; }
.smcap {font-variant: small-caps;}
.u {text-decoration: underline;}
.bold {font-weight: bold;}
.big { font-size: larger; }
.small { font-size: smaller; }
langcode:

category:
Kantai Collection, NSFW
siteabbrev:
qq
publisher:
forum.questionablequesting.com
forumtags:
Azur Lane, Bukkake, Impregnation/Breeding Will Happen, Kantai Collection, Lactation, Lewd And Smut Ahead, Romance, Ruined Orgasm, Tentacles Gotta Tentacle
titleHTML:
(link removed)
numWords:
786,779
title:
Tails of the Reluctant Teitoku
tagsfromtitledetect:
[NSFW]Tails of the Reluctant Teitoku (Kantai Collection)
parentforums:
NSFW
warnings:

datePublished:
2019-07-07 13:44
authorId:
magisteraurelius.38041
version:
FanFicFare v4.0.2
genre:
Azur Lane, Bukkake, Impregnation/Breeding Will Happen, Kantai Collection, Lactation, Lewd And Smut Ahead, Romance, Ruined Orgasm, Tentacles Gotta Tentacle
language:

storyUrl:
(link removed)
authorHTML:
(link removed)
sectionUrl:
(link removed)
seriesHTML:

lastupdate:
Last Update Year/Month: 2021/02, Last Update: 2021/02/26
status:

formatext:
.epub
cover_image:
first
author:
MagisterAurelius
extratags:
FanFiction
authorUrl:
(link removed)
formatname:
epub
series:

dateCreated:
2021-02-28 19:36:54
words_added:

description:
<div class="messageText SelectQuoteContainer ugc baseHtml"> So, I've finally gotten around to starting a story of someone who normally would never get to be a harem protagonist. This is the story of him getting dragged kicking and screaming into the craziness that is a Teitoku of ship-girls. <br/> <br/>
A couple of things to remind readers of. In this Kancolle-verse, all ship-girls of destroyer class and above are physically mature and are physiologically and mentally eighteen years of age and/o</div>
seriesUrl:

rating:

storyId:
9938
numChapters:
114
tagsfromtitle:
Kantai Collection, NSFW
characters:

site:
forum.questionablequesting.com
username:
(was set, removed for security)
end:
None
site:
forum.questionablequesting.com

any idea why I'm gaeeting this kind of error with QQ SB & SV at times?

[JimmXinu]

You're running out of memory when FFF tries to give Calibre the updated/downloaded epub. I suspect that epub is huge due to images.

The version of Calibre you are using is more than 3 years old and isn't even the last v3 variant. I suggest upgrading using the official Calibre download.

If you are using Windows and the 32 bit version, I also suggest using the 64 bit version of Calibre instead.

calibre, version 3.14.0
...
Error: out of memory

[J3L]

Any idea why for the last several weeks I am unable to download from SOL?
For instance
https://storiesonline.net/s/75558/once-upon-an-alien
gives this error message

[JimmXinu]

That's a 404 page not found error. I'm able to download that story fine.

So either the site is flakey, or that's how they are choosing to implement throttling if you are downloading too fast.

I'd try setting a sleep time since that site doesn't have one in defaults:

[storiesonline.net]
slow_down_sleep_time:2

[ownedbycats]

Question: Why would FanFicFare ask me to change the story URL when it's identical to the new one? URL encoding, perhaps? But it was an AO3 url, no odd characters.

[JimmXinu]

Without seeing the before and after URLs, I don't know. If you use logpage, does it show a URL change?

[J3L]

Originally Posted by JimmXinu

That's a 404 page not found error. I'm able to download that story fine.

So either the site is flakey, or that's how they are choosing to implement throttling if you are downloading too fast.

I'd try setting a sleep time since that site doesn't have one in defaults:

[storiesonline.net]
slow_down_sleep_time:2

Thank you. But that doesn't work either.

Where does FFF keep personal settings so I can start with a clean slate? Just updating the plug-in keeps personal settings/data.

[ownedbycats]

No logpage. Actually I think there might've been a trailing slash on one of the URLs that I failed to notice.

[JimmXinu]

Originally Posted by J3L

Thank you. But that doesn't work either.

Where does FFF keep personal settings so I can start with a clean slate? Just updating the plug-in keeps personal settings/data.

In FFF's config, under the 'Other' tab, there's a button for 'View Library preferences'.

If you select 'settings' in the left pane, you can use the 'Clear' button at the bottom to return to all default settings.

(Assuming it still works--it's not something I test very often.)