[GUI Plugin] FanFictionDownLoader - Page 41

rlh3 · 07-15-2012, 04:07 AM

@JimmXimu & @iatheia

Works fine! Thank you very much.

Firedancer885 · 07-15-2012, 08:49 AM

Quote:

Originally Posted by JimmXinu

New test version with Count Pages integrate now attached to the earlier post.

Let me know how it works for you.

Thanks

Works without problems.

JimmXinu · 07-15-2012, 12:00 PM

Version 1.5.44 - 15 Jul 2012

New Site: finestories.com
New Site: www.hpfanficarchive.com
New Site: svufiction.com
New Site: www.twilightarchives.com
New Site: www.wizardtales.net

New Feature: Multiple authors per story. Currently supported for AO3, TtH and wraithbait.com. More will be added as we become aware that they use multiple author.

New Plugin Feature: Option to call 'Count Pages' plugin on new/update. Requires Count Pages 1.6.0 or better.

Fixes for latest fanfiction.net change.

rlh3 · 07-15-2012, 05:04 PM

Well, here I am trying to break things...

In the finestories.com adapter I am having a problem with some stories. When the story is broken into sections I get an error: 'NoneType' object has no attribute 'text'. The two stories in question are found at: http://finestories.com/s/10437 and http://finestories.com/s/10288.

I hope that this in enough information to locate the difficulty.

iatheia · 07-15-2012, 05:45 PM

Ah, it appears that the categories are optional, after all.

Fix in the next update

JimmXinu · 07-15-2012, 06:48 PM

@rlh3 - Try this version.

rlh3 · 07-16-2012, 12:43 AM

I had an install error, but it was my own fault, I will check out teh update and report back.

rlh3 · 07-16-2012, 12:59 AM

After a stupid install error everything is working fine.

Thank you very much.

JimmXinu · 07-16-2012, 09:59 PM

Ida's most recent update: "Yet another fix for fnst"

Also fix for img in thewriterscoffeeshop.com title.

cryzed · 07-18-2012, 10:31 AM

Quote:

Originally Posted by JimmXinu

I prefer not to draw attention in case they try and block us.

If I recall correctly their guidelines didn't explicitly forbid the usage of such scripts... or well they could be interpreted in several ways. For example:

C. You agree not to access User Submissions (defined below) or FanFiction.Net Content through any technology or means other than the Website itself.
E. You agree not to use or launch any automated system, including without limitation, "robots," "spiders," or "offline readers," that accesses the Website in a manner that sends more request messages to the FanFiction.Net servers in a given period of time than a human can reasonably produce in the same period by using a conventional on-line web browser.

C: Lemon or the FanFictionDownLoader Calibre plugin in fact does access the content over the website (by sending a HTTP request, not for example somehow accessing the underlying databases) so in a loose sense they are also browsers and simply display content out of their "cache" which is saved locally and E: Calibre only uses one thread to retrieve content as far as I know -- thus it doesn't fall into the categorization "'robots', 'spiders', or 'offline readers', that accesses the Website in a manner that sends more request messages to the FanFiction.Net servers in a given period of time than a human can reasonably produce in the same period by using a conventional on-line web browser." I could potentially just open every single chapter within a short amount of time and read them by changing tabs.

Well I'm probably too naive judging this though, so not drawing attention is the best course of action I assume. I once sent them an e-mail regarding Lemon and explained to them that it wouldn't cause much more stress on their servers if the users didn't change the number of concurrent threads to something insane and that I would continue interpreting the terms in the way I just explained unless they told me to stop or further clarified their terms of service: they never answered -- so I don't really feel bad about it.

Also Jimm: I really like the current Count Pages plugin integration, but is it possible to call the plugin in such a way that it automatically applies the metadata without additionally having to confirm the dialogue that pops up?

JimmXinu · 07-18-2012, 11:04 AM

Pragmatically, I interpret ffnet's TOS as "we recognize we can't realistically forbid 'bots, but we reserve the right to block them if they get out of hand". And we have evidence from one of the other developers that they have blocked IPs for excessive traffic in past.

FFDL has a hard coded slow-down for ffnet downloads in an effort to not get out of hand. I've never heard of anyone getting blocked using it, so I assume it's slow enough.

But I'm still paranoid enough that I'd prefer not to take chances on drawing their attention.

Firedancer885 · 07-18-2012, 03:51 PM

Quote:

Originally Posted by cryzed

Also Jimm: I really like the current Count Pages plugin integration, but is it possible to call the plugin in such a way that it automatically applies the metadata without additionally having to confirm the dialogue that pops up?

That would be good, but I think it's mainly a Count Pages problem, not an FFDL one.

JimmXinu · 07-18-2012, 04:05 PM

@cryzed - Editing past posting doesn't generate a notice. So if Firedancer885 hadn't responded, I'd have never seen your question, since I'd already read that post.

Quote:

Originally Posted by Firedancer885

That would be good, but I think it's mainly a Count Pages problem, not an FFDL one.

Correct. If kiwidude puts a parameter on count_statistics() to bypass the 'Proceed with updating columns in your library?' dialog I'll be happy to call it that way.

kiwidude has posted on this thread in past--not sure if he monitors it or if somebody PM'd him.

theducks · 07-18-2012, 04:15 PM

Quote:

Originally Posted by JimmXinu

@cryzed - Editing past posting doesn't generate a notice. So if Firedancer885 hadn't responded, I'd have never seen your question, since I'd already read that post.

Correct. If kiwidude puts a parameter on count_statistics() to bypass the 'Proceed with updating columns in your library?' dialog I'll be happy to call it that way.

kiwidude has posted on this thread in past--not sure if he monitors it or if somebody PM'd him.

One of the reasons for the confirmation, is Kiwidudes plugins are not Modal (Exclusive until done). They need to know when it is OK to update the Library metadata. The choice was lock you out till done (could be hours) or run in background and click OK. I like the second choice

kiwidude · 07-18-2012, 04:20 PM

@Jimm - how much I monitor depends on whether I am waiting for something on my machine (like a metadata download)...

I'm not likely to add such a parameter. Count Pages is a job which runs in the background of calibre, just like downloading metadata. At the point that it has the information ready to update your books, it *must* have exclusive control of the calibre database. Imagine if you were allowed to be editing one of those books while calibre is also trying to update the database for those books - you have the whole "hell" of concurrency and stale data to deal with.

You have to suck it up and live with it basically

07-15-2012, 06:48 PM	#606
JimmXinu Plugin Developer Posts: 6,980 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	@rlh3 - Try this version. Last edited by JimmXinu; 07-16-2012 at 09:57 PM. Reason: Remove obsolete beta versions

07-16-2012, 09:59 PM	#609
JimmXinu Plugin Developer Posts: 6,980 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	Ida's most recent update: "Yet another fix for fnst" Also fix for img in thewriterscoffeeshop.com title. Last edited by JimmXinu; 07-19-2012 at 09:02 PM. Reason: Remove obsolete beta versions

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Resize Cover	kiwidude	Plugins	100	07-10-2025 08:50 AM
[GUI Plugin] Count Pages	kiwidude	Plugins	1844	05-25-2025 11:24 AM
[GUI Plugin] Find Duplicates	kiwidude	Plugins	1124	04-18-2025 09:19 AM
[GUI Plugin] Open With	kiwidude	Plugins	404	02-21-2025 05:42 AM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

07-15-2012, 04:07 AM	#601
rlh3 Connoisseur Posts: 57 Karma: 10 Join Date: Jul 2012 Device: iPad	@JimmXimu & @iatheia Works fine! Thank you very much.

07-15-2012, 12:00 PM	#603
JimmXinu Plugin Developer Posts: 6,980 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	Version 1.5.44 - 15 Jul 2012 New Site: finestories.com New Site: www.hpfanficarchive.com New Site: svufiction.com New Site: www.twilightarchives.com New Site: www.wizardtales.net New Feature: Multiple authors per story. Currently supported for AO3, TtH and wraithbait.com. More will be added as we become aware that they use multiple author. New Plugin Feature: Option to call 'Count Pages' plugin on new/update. Requires Count Pages 1.6.0 or better. Fixes for latest fanfiction.net change.

07-15-2012, 05:04 PM	#604
rlh3 Connoisseur Posts: 57 Karma: 10 Join Date: Jul 2012 Device: iPad	Well, here I am trying to break things... In the finestories.com adapter I am having a problem with some stories. When the story is broken into sections I get an error: 'NoneType' object has no attribute 'text'. The two stories in question are found at: http://finestories.com/s/10437 and http://finestories.com/s/10288. I hope that this in enough information to locate the difficulty.

07-15-2012, 05:45 PM	#605
iatheia Zealot Posts: 139 Karma: 21754 Join Date: Mar 2011 Device: Kindle 3, Icarus Illumina XL, Boyue T62	Ah, it appears that the categories are optional, after all. Fix in the next update

07-16-2012, 12:43 AM	#607
rlh3 Connoisseur Posts: 57 Karma: 10 Join Date: Jul 2012 Device: iPad	I had an install error, but it was my own fault, I will check out teh update and report back.

07-16-2012, 12:59 AM	#608
rlh3 Connoisseur Posts: 57 Karma: 10 Join Date: Jul 2012 Device: iPad	After a stupid install error everything is working fine. Thank you very much.

07-18-2012, 11:04 AM	#611
JimmXinu Plugin Developer Posts: 6,980 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	Pragmatically, I interpret ffnet's TOS as "we recognize we can't realistically forbid 'bots, but we reserve the right to block them if they get out of hand". And we have evidence from one of the other developers that they have blocked IPs for excessive traffic in past. FFDL has a hard coded slow-down for ffnet downloads in an effort to not get out of hand. I've never heard of anyone getting blocked using it, so I assume it's slow enough. But I'm still paranoid enough that I'd prefer not to take chances on drawing their attention.

07-18-2012, 04:20 PM	#615
kiwidude Calibre Plugins Developer Posts: 4,730 Karma: 2197770 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	@Jimm - how much I monitor depends on whether I am waiting for something on my machine (like a metadata download)... I'm not likely to add such a parameter. Count Pages is a job which runs in the background of calibre, just like downloading metadata. At the point that it has the information ready to update your books, it must have exclusive control of the calibre database. Imagine if you were allowed to be editing one of those books while calibre is also trying to update the database for those books - you have the whole "hell" of concurrency and stale data to deal with. You have to suck it up and live with it basically

Advert

Advert