[GUI Plugin] FanFicFare - Page 5

Tanjamuse · 04-19-2015, 11:23 AM

Quote:

Originally Posted by JimmXinu

They are: [\^$.|?*+()

I'm adding something to the wiki page about replace_metadata about this.

If I have metadata that needs replacing, should I place a \ in front of alle of these characters?

JimmXinu · 04-19-2015, 11:25 AM

Quote:

Originally Posted by Tanjamuse

What am I doing wrong?

Well, for one thing, you're spamming the forum.

Please do not ask your question over and over again just because you didn't get an immediate response.

thatblondelady · 04-19-2015, 02:42 PM

Quote:

Originally Posted by JimmXinu

...
@thatblondelady: Good guess about the regex escaping, but & isn't a regex special character. They are: [\^$.|?*+()

I'm adding something to the wiki page about replace_metadata about this.

Yup, I confused bash script and regex in my head. I'm not amazing at either, obviously. Thank you!

JimmXinu · 04-20-2015, 12:15 AM

FYI, storiesonline.net has changed their HTML (back?) again. I'll look into it sometime tomorrow.

AndersW · 04-20-2015, 02:42 PM

I have recently been getting Incomplete Read errors when trying to update several stories. And it only started recently.

Quote:

IncompleteRead(1592 bytes read, 325 more expected)

I am trying to update from fimfiction.net, and it isn't unique to any story. It also seems to hang when it runs into this error when fetching metadata.

JimmXinu · 04-20-2015, 02:45 PM

Quote:

Originally Posted by JimmXinu

FYI, storiesonline.net has changed their HTML (back?) again. I'll look into it sometime tomorrow.

Test version posted for latest storiesonline.net changes.

Pay particular attention to be sure no story parts are being cut off at the end of chapters/pages.

JimmXinu · 04-20-2015, 03:03 PM

Quote:

Originally Posted by AndersW

I have recently been getting Incomplete Read errors when trying to update several stories. And it only started recently.

Code:

 IncompleteRead(1592 bytes read, 325 more expected)

I am trying to update from fimfiction.net, and it isn't unique to any story. It also seems to hang when it runs into this error when fetching metadata.

Even if you don't think it's associated with a particular story, please include a story (or a few) that you've had the problem with when reporting issues.

I tried 5 random FimF stories from the recent updates list without any problems.

That error indicates that you didn't receive all the bytes the response header indicated should arrive. It's most commonly causes are server side error or networking hardware error. And in fact FimF had a DDoS attack recently.

FanFicFare already has built in a retry mechanism when it has failures. I don't think there's anything else I can do about it.

fathertyme · 04-20-2015, 05:12 PM

Quote:

Originally Posted by JimmXinu

Test version posted for latest storiesonline.net changes.

Pay particular attention to be sure no story parts are being cut off at the end of chapters/pages.

storiesonline is being funky...

I'm getting some stories that download and then nonetype for others

eg:

success: http://storiesonline.net/s/11879

fail: http://storiesonline.net/s/72851
fail: http://storiesonline.net/s/11087

edit: at least its consistently inconsistent... if it gives the "NoneType" error, it always does it for that story...

AndersW · 04-21-2015, 12:47 AM

Quote:

Originally Posted by JimmXinu

Even if you don't think it's associated with a particular story, please include a story (or a few) that you've had the problem with when reporting issues.

I tried 5 random FimF stories from the recent updates list without any problems.

That error indicates that you didn't receive all the bytes the response header indicated should arrive. It's most commonly causes are server side error or networking hardware error. And in fact FimF had a DDoS attack recently.

FanFicFare already has built in a retry mechanism when it has failures. I don't think there's anything else I can do about it.

A sample of 880 stories produced 57 stories that had that error, about 1 in 15. Dropping that list of 57 into the downloader resulted in 8 stories with that error, about 1 in 7. So yes some stories are more prone to it than others. Therefore I will leave you with the list of 57 that has higher odds of running into the error. It also seemed slightly less common than was happening this morning.

Spoiler:

Code:

http://www.fimfiction.net/story/246808/
http://www.fimfiction.net/story/254829/
http://www.fimfiction.net/story/257272/
http://www.fimfiction.net/story/204498/
http://www.fimfiction.net/story/249920/
http://www.fimfiction.net/story/256907/
http://www.fimfiction.net/story/256915/
http://www.fimfiction.net/story/256505/
http://www.fimfiction.net/story/253518/
http://www.fimfiction.net/story/259902/
http://www.fimfiction.net/story/32725/
http://www.fimfiction.net/story/259539/
http://www.fimfiction.net/story/258667/
http://www.fimfiction.net/story/259231/
http://www.fimfiction.net/story/252120/
http://www.fimfiction.net/story/38132/
http://www.fimfiction.net/story/244347/
http://www.fimfiction.net/story/260328/
http://www.fimfiction.net/story/258929/
http://www.fimfiction.net/story/260761/
http://www.fimfiction.net/story/260450/
http://www.fimfiction.net/story/236801/
http://www.fimfiction.net/story/122018/
http://www.fimfiction.net/story/250351/
http://www.fimfiction.net/story/260030/
http://www.fimfiction.net/story/259064/
http://www.fimfiction.net/story/259245/
http://www.fimfiction.net/story/259766/
http://www.fimfiction.net/story/227220/
http://www.fimfiction.net/story/141090/
http://www.fimfiction.net/story/259171/
http://www.fimfiction.net/story/251013/ --- Did not list how many bytes were missing
http://www.fimfiction.net/story/250387/ --- Did not list how many bytes were missing
http://www.fimfiction.net/story/28840/ --- Did not list how many bytes were missing
http://www.fimfiction.net/story/103084/
http://www.fimfiction.net/story/230473/
http://www.fimfiction.net/story/189831/
http://www.fimfiction.net/story/204043/
http://www.fimfiction.net/story/258870/
http://www.fimfiction.net/story/240032/
http://www.fimfiction.net/story/257601/
http://www.fimfiction.net/story/257327/
http://www.fimfiction.net/story/235864/
http://www.fimfiction.net/story/107689/
http://www.fimfiction.net/story/188767/
http://www.fimfiction.net/story/258607/
http://www.fimfiction.net/story/240191/
http://www.fimfiction.net/story/255079/
http://www.fimfiction.net/story/258000/
http://www.fimfiction.net/story/223153/
http://www.fimfiction.net/story/147292/
http://www.fimfiction.net/story/239743/
http://www.fimfiction.net/story/257494/
http://www.fimfiction.net/story/257889/
http://www.fimfiction.net/story/257878/
http://www.fimfiction.net/story/253819/
http://www.fimfiction.net/story/186493/

Hopefully this will help.

JimmXinu · 04-21-2015, 02:09 AM

Quote:

Originally Posted by AndersW

A sample of 880 stories produced 57 stories that had that error, about 1 in 15. Dropping that list of 57 into the downloader resulted in 8 stories with that error, about 1 in 7.

A slightly updated quote from one of my own earlier posts on the old thread:

Quote:

First, I do not recommend using FanFicFare the way you are. Checking for updates on hundreds of stories at once causes excessive traffic on the source site. fanfiction.net in particular has been known to block people's IP address if they generate too much traffic.

Instead, I highly recommend that you 'follow' stories and use the sites' email notifications to inform you about updates and then update those specific stories. I also recommend using Update EPUB, not Overwrite, unless you have reason to believe that a particular story has changes in previous chapters.

I must repeat, doing mass updates like this is not recommended.

(I acknowledge that I should make an FAQ for this.)

I consider 15 stories to be about the upper limit of how many a user should update at once with any regularity. 57 is really pushing what I think is reasonable for once-in-a-while updates. 880 at once: if it was my website, I'd likely be blocking you.

I'm not going to download 57 stories to test this. I think it's most likely you are bumping into--or causing--transient errors on the server side. Plus, I still don't think there's anything else I could do about it.

Tanjamuse · 04-21-2015, 11:07 AM

Can I create a Generate Cover setting based on what is in/not in a user category?

JimmXinu · 04-21-2015, 11:48 AM

Quote:

Originally Posted by fathertyme

storiesonline is being funky...
...
edit: at least its consistently inconsistent... if it gives the "NoneType" error, it always does it for that story...

There are two places in the code that look for the same 'story' element. I'd only updated one of them in my change. So that's my bad.

Test version posted for latest storiesonline.net changes.

AndersW · 04-21-2015, 12:28 PM

Quote:

Originally Posted by JimmXinu

A slightly updated quote from one of my own earlier posts on the old thread:

(I acknowledge that I should make an FAQ for this.)

I consider 15 stories to be about the upper limit of how many a user should update at once with any regularity. 57 is really pushing what I think is reasonable for once-in-a-while updates. 880 at once: if it was my website, I'd likely be blocking you.

I'm not going to download 57 stories to test this. I think it's most likely you are bumping into--or causing--transient errors on the server side. Plus, I still don't think there's anything else I could do about it.

Cutting my list up into chunks of 15 still resulted in this error occurring about 1 in 7 times.

Spoiler:

Does FanFicFare trigger a retry when it receives an incomplete read error? That is the only thing I could think of. I think it should do that, but I just want to check if it is.

JimmXinu · 04-21-2015, 02:15 PM

@AndersW - I had somewhat similar errors at one time when the ether switch in my closet was starting to go bad--so it could be network hardware. But it affected more than just the downloader.

I did try your first chunk of 15 and they all worked fine for me. If you did several sets of 15 in a row, that's nearly the same as doing them all at once.

Quote:

Originally Posted by AndersW

Does FanFicFare trigger a retry when it receives an incomplete read error? That is the only thing I could think of. I think it should do that, but I just want to check if it is.

Quote:

Originally Posted by JimmXinu

...
FanFicFare already has built in a retry mechanism when it has failures. I don't think there's anything else I can do about it.

JimmXinu · 04-21-2015, 02:31 PM

- Even more fix for storiesonline.net site changes--plus switch to BS4.

Test version posted.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	527	08-15-2025 01:36 PM
[GUI Plugin] Open With	kiwidude	Plugins	404	02-21-2025 05:42 AM
[GUI Plugin] Marvin XD	Philantrop	Plugins	126	01-29-2017 12:48 PM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

04-20-2015, 12:15 AM	#64
JimmXinu Plugin Developer Posts: 7,127 Karma: 5005503 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	FYI, storiesonline.net has changed their HTML (back?) again. I'll look into it sometime tomorrow.

04-21-2015, 11:07 AM	#71
Tanjamuse Wizard Posts: 1,327 Karma: 5306 Join Date: Jan 2014 Device: none	Can I create a Generate Cover setting based on what is in/not in a user category?

04-21-2015, 02:31 PM	#75
JimmXinu Plugin Developer Posts: 7,127 Karma: 5005503 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	- Even more fix for storiesonline.net site changes--plus switch to BS4. Test version posted.