[GUI Plugin] FanFictionDownLoader

[Firedancer885]

Originally Posted by JimmXinu

Firedancer885 -- Never mind, I've posted it now.

Thanks. I only realized the problem after you posted the 02 version of the plugin. Is there a way to weed out the affected authors? I have 1966 authors in my library, AND I managed to go on a Hansel and Gretel binge last night. I took on at least 10 new authors beside the authors whose stories I updated.

[JimmXinu]

Originally Posted by Tanjamuse

I don't understand the website encodings?

## website encoding(s) In theory, each website reports the character
## encoding they use for each page.  In practice, some sites report it
## incorrectly.  Each adapter has a default list, usually "utf8,
## Windows-1252" or "Windows-1252, utf8", but this will let you
## explicitly set the encoding and order if you need to.  The special
## value 'auto' will call chardet and use the encoding it reports if
## it has +90% confidence.  'auto' is not reliable.
website_encodings:Windows-1252,ISO-8859-1,auto

Look in your personal.ini and see if you have a website_encodings parameter for [defaults], [archiveofourown.org] or [overrides].

If you don't already have one, try setting one like the one above under [archiveofourown.org].

Originally Posted by Tanjamuse

It's getting quite annoying with having to try to download some stories 4-5 times and still never get them.

You will get the best help when you post the story URLs you are having problems with, and the debug output. The one story URL you did post, I tried and had no problems with using any of the common encodings.

And AO3 has built-in ebook download support. If you cannot resolve your issues, you can always download epubs and other ebook formats directly from AO3.

Originally Posted by Tanjamuse

And I've tried reading that link, you sent me but I don't understand any of it.

Escaping special characters in regular expressions has been explained to you before.

If you want to use advanced features, some advanced knowledge may be required.

[BookJunkieLI]

Originally Posted by Firedancer885

Thanks. I only realized the problem after you posted the 02 version of the plugin. Is there a way to weed out the affected authors? I have 1966 authors in my library, AND I managed to go on a Hansel and Gretel binge last night. I took on at least 10 new authors beside the authors whose stories I updated.

Wouldn't selecting Anonymous in Author section of the tag browser give you all of the affected the stories? You could then run FFDL using Update Metadata Only on just those stories to get the correct author. If you want to make sure that it only updates the Author then you could go into configure FFDL and mark all of the Standard and Custom columns except Author to New Only. You just need to make sure you uncheck them afterwards.

[Firedancer885]

Originally Posted by BookJunkieLI

Wouldn't selecting Anonymous in Author section of the tag browser give you all of the affected the stories? You could then run FFDL using Update Metadata Only on just those stories to get the correct author. If you want to make sure that it only updates the Author then you could go into configure FFDL and mark all of the Standard and Custom columns except Author to New Only. You just need to make sure you uncheck them afterwards.

Thanks, that's not a problem any more. I was referring to the author url problem I reported after Jimm posted the official fix for the anonymous author problem.

[thecookiemommma]

Originally Posted by JimmXinu

And I'm frankly getting tired of the fact that all the requests for new sites lately seem to be porn.

However, if somebody else wants to write the code to support it, I will include it.

If it helps, for sites like that that don't fit the bill (and LJ, etc), I use a Firefox extension called "GrabMyBooks" that grabs selected text and makes an ebook out of them. That might be something worth considering for those weird ones.

[Firedancer885]

Originally Posted by thecookiemommma

If it helps, for sites like that that don't fit the bill (and LJ, etc), I use a Firefox extension called "GrabMyBooks" that grabs selected text and makes an ebook out of them. That might be something worth considering for those weird ones.

Thank you sooo much for that suggestion.

[BookJunkieLI]

Originally Posted by Firedancer885

Thanks, that's not a problem any more. I was referring to the author url problem I reported after Jimm posted the official fix for the anonymous author problem.

Ah. I see what you're talking about now. The author URL isn't something I typically pay much attention too. Assuming you haven't figured out how to fix it yourself, if you have feel free to disregard, here's what i came up with.

Poking around a bit, it seems like your best option to fix it would be to sort your stories by Date added to Calibre. Actually, I'd sort by Author then Date so that the stories will be grouped by Author for each date. Since you know it would only affect stories added in this past week you can then select one story from each affected author and run the Update Metadata again. This should fix the author URL issue without having to go into Manage Authors and do it manually for each author.

[Firedancer885]

Originally Posted by BookJunkieLI

Ah. I see what you're talking about now. The author URL isn't something I typically pay much attention too. Assuming you haven't figured out how to fix it yourself, if you have feel free to disregard, here's what i came up with.

...

Thanks

[BookJunkieLI]

Welcome.

[FicReader81]

Hi. Since the last update of the plugin, I have started to receive the following error when trying to download/update stories from literotica.

invalid literal for int() with base 10: ''

Can you please tell me how to fix this problem? Reply/support will be appreciated.

[JimmXinu]

Originally Posted by FicReader81

Hi. Since the last update of the plugin, I have started to receive the following error when trying to download/update stories from literotica.

invalid literal for int() with base 10: ''

Can you please tell me how to fix this problem? Reply/support will be appreciated.

doe5716 made a number of changes to improve the literotica adapter.

I don't read that site, so I haven't been paying much attention to the changes.

If you post the story URL and/or a debug log I'll look into it.

[FicReader81]

Originally Posted by JimmXinu

doe5716 made a number of changes to improve the literotica adapter.

I don't read that site, so I haven't been paying much attention to the changes.

If you post the story URL and/or a debug log I'll look into it.

It is happening about 8/10 times. As an example, for the story with URL :

http://www.literotica.com/s/youll-find-anything-online

Is this the debug log? I am not sure, if its not this then please tell me how to get it.

calibre Debug log
calibre 1.48 isfrozen: True is64bit: False
Windows-Vista-6.0.6002-SP2 Windows ('32bit', 'WindowsPE')
('Windows', 'Vista', '6.0.6002')
Python 2.7.5
Windows: ('Vista', '6.0.6002', 'SP2', 'Multiprocessor Free')
Starting up...
FFDLEBUG:calibre_plugins.fanfictiondownloader_pl ugin.ffdl_plugin(158):Plugin FanFictionDownLoader macmenuhack file_path:C:\Users\HP\AppData\Roaming\calibre\plug ins\fanfictiondownloader_macmenuhack.txt
FFDLEBUG:calibre_plugins.fanfictiondownloader_pl ugin.ffdl_plugin(734):FanFictionDownLoader v2.0.3
FFDLEBUG:calibre_plugins.fanfictiondownloader_pl ugin.ffdl_plugin(769):url:http://www.literotica.com/s/youll-find-anything-online
FFDLEBUG:calibre_plugins.fanfictiondownloader_pl ugin.fanficdownloader.adapters.adapter_literotica( 121):Chapter/Story URL: <http://www.literotica.com/s/youll-find-anything-online>
FFDLEBUG:calibre_plugins.fanfictiondownloader_pl ugin.ffdl_plugin(935):update existing id:2735
FFDL:ERROR:calibre_plugins.fanfictiondownloader_pl ugin.dialogs(611):Exception: {u'comment': u"invalid literal for int() with base 10: ''", u'status': u'Add', u'pubdate': datetime.datetime(2014, 8, 6, 0, 0, tzinfo=SafeLocalTimeZone()), u'listorder': 0, u'site': 'literotica.com', u'added': False, u'end': None, u'author': [u'SDLRob'], u'comments': u"<p>Alexis is caught enjoying Kate's pornstar debut.</p>", u'calibre_id': 2735, u'series': '', u'username': 'NoneGiven', u'begin': None, u'good': False, u'tags': ['FanFiction'], u'timestamp': datetime.datetime(2014, 8, 16, 3, 21, 14, 781000, tzinfo=SafeLocalTimeZone()), u'is_adult': False, u'updatedate': datetime.datetime(2014, 8, 6, 0, 0, tzinfo=SafeLocalTimeZone()), u'all_metadata': {'rating': '', 'eroticatags': u'Celebrities', 'series': '', 'numWords': '', 'site': u'literotica.com', 'datePublished': u'2014-08-06', 'authorHTML': u"<a class='authorlink' href='http://www.literotica.com/stories/memberpage.php?uid=1801739&page=submissions'>SDLRo b</a>", 'storyUrl': u'http://www.literotica.com/s/youll-find-anything-online', 'storyId': u'youll-find-anything-online', 'category': u'', 'lastupdate': u'Last Update Year/Month: 2014/08, Last Update: 2014/08/06', 'formatext': '', 'author': u'SDLRob', 'ships': u'', 'seriesHTML': '', 'formatname': '', 'version': u'FanFictionDownLoader v2.0.3', 'dateUpdated': u'2014-08-06', 'numChapters': '', 'status': '', 'langcode': '', 'description': u"Alexis is caught enjoying Kate's pornstar debut.", 'warnings': u'', 'dateCreated': u'2014-08-16 03:21:14', 'output_css': '', 'authorId': u'1801739', 'authorUrl': u'http://www.literotica.com/stories/memberpage.php?uid=1801739&page=submissions', 'characters': u'', 'genre': u'', 'language': '', 'seriesUrl': '', 'siteabbrev': u'litero', 'title': u"You'll Find Anything Online", 'extratags': u''}, u'password': '', u'icon': u'plus.png', u'publisher': u'literotica.com', u'url': u'http://www.literotica.com/s/youll-find-anything-online', u'author_sort': [u'SDLRob'], u'title': u"You'll Find Anything Online"}:invalid literal for int() with base 10: ''
Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.dialo gs", line 601, in do_loop
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1037, in prep_download_loop
ValueError: invalid literal for int() with base 10: ''

Thank you for the reply.

[FicReader81]

Debug log in Txt format.

[JimmXinu]

Originally Posted by FicReader81

Is this the debug log? I am not sure, if its not this then please tell me how to get it.

Yep, that's exactly what I wanted. The attached .txt file was fine. When including in the post directly, wrapping it in CODE tags helps with readability.

The adapter wasn't setting the number of chapters correctly all the time. The attached version seems to work better for me.

[FicReader81]

Originally Posted by JimmXinu

Yep, that's exactly what I wanted. The attached .txt file was fine. When including in the post directly, wrapping it in CODE tags helps with readability.

The adapter wasn't setting the number of chapters correctly all the time. The attached version seems to work better for me.

Thanks. It seems to have fixed the problem. I really appreciate the instant help. Thank you once again.