[GUI Plugin] FanFictionDownLoader

[citac]

KayLee's question just prompted me to check the list of unsupported sites because I was thinking about asking whether http://www.blackraptor.net/m7fic/ can be supported. Since it seems to be a website with html and txt stories, that would make it unlikely to be supported, yes?

[JimmXinu]

@citac - Yes, sorry. Such sites tend to have *mostly* consistent formatting that causes parsing problems when you run into the inconsistent cases. And FFDL expects HTML, not text.

[citac]

@JimmXinu, ah well, I presumed that would be so but thought I should ask just in case. A lot of the authors I follow who used to post there have now switched to AO3, so that's fine. For those few that haven't, I'm willing to do the conversion myself every once in a while. Thanks!

[KayLee]

How do I get fics from adult-fan-fiction.net I know it should be able to do that being that it works just like fanfiction.net?

[JimmXinu]

Originally Posted by KayLee

How do I get fics from adult-fan-fiction.net I know it should be able to do that being that it works just like fanfiction.net?

There is no site at adult-fan-fiction.net.

If you're talking about adultfanfiction.net, it's documented that we don't support that site, and why:

None of the current developers are interested in writing code to support it because of the ominous legal warnings and requirements to access the site

adultfanfiction.net

[Firedancer885]

Jimm,

please check this tth url:

http://www.tthfanfic.org/Story-29169...vans+Story.htm

I get the following error when I try to update the story.

Bad
Flowers and Time: An Evans' Story
smolder
'NoneType' object has no attribute '__getitem__'
http://www.tthfanfic.org/Story-29169

It got quarantined by a site moderator, but I want to save it.

[JimmXinu]

@Firedancer885 - It's working fine for me now.

I've been following that story, too. They'd posted a 27th chapter that was just a hiatus message. That's now been moved to the end of chapter 26 instead. I expect that was why it was quarantined.

If anyone has an example of a quarantined story, I can add better support for it. I don't know what a quarantined story looks like.

[JimmXinu]

Version 1.7.28 - 22 Jun 2013

• Minor fixes to plugin
• Fix for siye.co.uk stories with HTML in the title

[thecookiemommma]

Just so people are aware, AdultFanFiction.net is supported by flagfic.com It's definitely more tedious than this solution, but if you've got a few stories you need (especially if they're done), that's a good option.

[KayLee]

Originally Posted by thecookiemommma

Just so people are aware, AdultFanFiction.net is supported by flagfic.com It's definitely more tedious than this solution, but if you've got a few stories you need (especially if they're done), that's a good option.

Thanks for the info

[RadiantPhoenix]

The plugin doesn't seem to work for me.

Does anyone have any idea what's going wrong and how to fix it?

I'm running Calibre "0.9.31+dfsg-1~ppa1" from https://launchpad.net/~n-muench/+archive/calibre2 on 64-bit Xubuntu 12.04, and have python version "2.7.3-0ubuntu2.2"

$ calibre

Plugin FanFictionDownLoader macmenuhack file_path:~/.config/calibre/plugins/fanfictiondownloader_macmenuhack.txt
FanFictionDownLoader v1.7.28
url:http://www.fanfiction.net/s/5398503/1/
FFDL:DEBUG:calibre_plugins.fanfictiondownloader_plugin.fanficdownloader.adapters.adapter_fanfictionnet(79):URL: http://www.fanfiction.net/s/5398503/1/
FFDL:DEBUG:calibre_plugins.fanfictiondownloader_plugin.fanficdownloader.adapters.adapter_fanfictionnet(112):=Trying newer chapter: http://www.fanfiction.net/s/5398503/83/
Exception: {u'comment': u"'function' object has no attribute 'Markdown'", u'publisher': u'www.fanfiction.net', u'begin': None, u'good': False, u'end': None, u'author': [u'Vathara'], u'url': u'http://www.fanfiction.net/s/5398503/1/', u'listorder': 0, u'author_sort': [u'Vathara'], u'all_metadata': {'rating': u'T', 'author': u'Vathara', 'series': None, 'numWords': u'713,824', 'site': u'www.fanfiction.net', 'datePublished': u'2009-09-24', 'authorHTML': u"<a class='authorlink' href='http://www.fanfiction.net/u/77482/Vathara'>Vathara</a>", 'storyUrl': u'http://www.fanfiction.net/s/5398503/1/', 'storyId': u'5398503', 'category': u'Avatar: Last Airbender', 'lastupdate': u'Last Update Year/Month: 2013/06, Last Update: 2013/06/06', 'formatext': None, 'title': u'Embers', 'ships': None, 'seriesHTML': None, 'formatname': None, 'version': u'FanFictionDownLoader v1.7.28', 'dateUpdated': u'2013-06-06', 'numChapters': u'82', 'status': u'In-Progress', 'langcode': u'en', 'description': u'Dragon\'s fire is not so easily extinguished; when Zuko rediscovers a lost firebending technique, shifting flames can shift the world... Follows "Theft Absolute".', 'warnings': None, 'dateCreated': u'2013-06-24 11:53:51', 'output_css': None, 'authorId': u'77482', 'authorUrl': u'http://www.fanfiction.net/u/77482/Vathara', 'characters': u'Iroh, Zuko', 'genre': u'Adventure, Family', 'favs': u'2,975', 'language': u'English', 'follows': u'2,321', 'seriesUrl': None, 'reviews': u'6,551', 'siteabbrev': u'ffnet', 'extratags': u'FanFiction'}, u'comments': u'', u'calibre_id': None, u'added': False, u'title': u'Embers', u'tags': [u'Iroh', u'Avatar: Last Airbender', u'Zuko', u'Family', u'In-Progress', u'FanFiction', u'Adventure']}:'function' object has no attribute 'Markdown'
Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.dialogs", line 527, in do_loop
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 772, in prep_download_loop
  File "/usr/lib/calibre/calibre/library/comments.py", line 137, in sanitize_comments_html
    md = markdown.Markdown(safe_mode=True)
AttributeError: 'function' object has no attribute 'Markdown'
Fetched metadata for 1 of 1

$ calibre --version
calibre (calibre 0.9.31)

$ python --version
Python 2.7.3

EDIT: the standalone CLI python program seems to run fine, but I was hoping to take easy advantage of the Calibre library stuff.

[JimmXinu]

@RadiantPhoenix -

This appears to be a problem with the function sanitize_comments_html inside calibre. FFDL has been using that function for 9 months without problems anywhere else. (The CLI version works because it isn't calling the calibre function.)

Are you the same user who posted this problem to the googlegroups.com list? If not, that would strongly suggest it's a problem with the calibre package you're using. If you are the same user, it could also suggest a problem with your install. You might try reinstalling calibre.

You could PM kovidgoyal (the primary calibre dev) about this is, but I expect his first response will also be to try the latest version of calibre.

[RadiantPhoenix]

Originally Posted by JimmXinu

@RadiantPhoenix -

This appears to be a problem with the function sanitize_comments_html inside calibre. FFDL has been using that function for 9 months without problems anywhere else. (The CLI version works because it isn't calling the calibre function.)

Are you the same user who posted this problem to the googlegroups.com list? If not, that would strongly suggest it's a problem with the calibre package you're using. If you are the same user, it could also suggest a problem with your install. You might try reinstalling calibre.

You could PM kovidgoyal (the primary calibre dev) about this is, but I expect his first response will also be to try the latest version of calibre.

I am not the same user, my googling couldn't find anyone else who had the problem.

I'll look into a different build of Calibre.

(I did manage to find the user post you were talking about after you told me about it, and if I'd seen it before I would have posted there.)

EDIT: Based on the other guy's post, my first guess is that the Calibre build in the PPA from four days ago is messed up somehow and the other person didn't notice because he didn't download anything between the 20th and the 23rd.

EDIT 2: It looks like the Precise repos don't have new enough versions of several necessary libraries for the latest version of calibre:

• PyQt has 4.9.1 wants 4.9.6
• poppler has 0.18.4 wants 0.20.2
• libmtp has 1.1.3 wants 1.1.5
• netifaces has 0.6 wants 0.8
• cssseleect has 0.6.1 wants 0.7.1

Is this correct?

EDIT 3: It appears to come with its own libraries.

EDIT 4: And, it works.

Problem solved.

I didn't need to do anything fancy, I just extracted the tarball and ran the 'calibre' script. It went just fine.

EDIT 5:

Okay, seriously, why is this thing not downloading cover images? I've tried everything I could think of or find on google in personal.ini.

[JimmXinu]

Originally Posted by RadiantPhoenix

I am not the same user, my googling couldn't find anyone else who had the problem.

...

I'm glad you were able to get it working. Thanks for explaining what worked for you, I'll pass it on to the other user.

[RadiantPhoenix]

So, how do I get cover images working?