[GUI Plugin] FanFictionDownLoader

[JimmXinu]

Originally Posted by pookakitten

I am using the most recent version (2.14) of FFDL and it keeps losing my settings. It loses my custom column (status). The personal.ini keeps being reset to default so I lose what tags I want saved. It removes my settings for the reading list plugin.

It's an install of Calibre, ver 2.20 on Windows 7. The only unusual thing about how I use it is I have a virtual library set up for my fan fiction.

Any idea why this is happening?

davidfor said the same thing I was going to. I will add a link to the discussion about the bug:

https://www.mobileread.com/forums/sho...d.php?t=256268

It was also previously discussed in this thread:

https://www.mobileread.com/forums/sho...postcount=3755

[cryzed]

This might be the right time to consider moving to GitHub.

[JimmXinu]

Originally Posted by cryzed

This might be the right time to consider moving to GitHub.

Well, that's a pain. Thanks for the heads up, I hadn't seen that yet.

[cryzed]

Yep, it will be... a pain, that is. It might also be an opportunity to grow and garner some more publicity with a platform that is as active and popular as GitHub. Of course that would require the conversion from the Mercurial repository to a Git repository -- but according to the linked article there should be tools to do exactly that. One might also want to consider starting anew and keep the Mercurial commit history somewhere saved separately and simply import all current files into a fresh Git repository.

The neat thing about GitHub is also that you can just put up releases for your repository, so the release *.zip file we currently carry around in the repository and its history could finally be removed, and the big, inefficient binary diffs it has caused over time would disappear too (assuming you plan to create an entirely new Git repository).

Well there's still a bit of time yet, and I don't want to force my ideas onto you again, but I think this would be the ideal time for some neat changes (excluding having to learn about Git).

[Czarcasm]

Did some testing on several computers with multiple connections and the problem seems to be Fimfiction.net. I don't know if something with the site changed that broke connectivity with the plugin or what. Yeah, I can download epubs directly from the site but I liked being able to update everything from Calibre.

[MGSaintz]

It's not just Fimfiction.net, I'm also having the same issues that Czarcasm is getting and it is also giving errors on downloading from tthfanfic.org and tenhawkpresents.com.

[JimmXinu]

If you guys can give me some story URLs it happens with, I might be able to see more. The only one given (http://www.fimfiction.net/story/13616/) worked fine for me. And I use tthfanfic.org pretty much every day without error.

So far, my diagnosis remains the same: some kind of network failure issue or, less likely, a problem with the web server.

[Czarcasm]

Grabbed a few random fics from Fimfiction and Tthfanfic and with the exception of one, they all had the same problem.

=Fimfiction=
http://www.fimfiction.net/story/2527...age-ascendancy
http://www.fimfiction.net/story/1305...s-of-equestria
http://www.fimfiction.net/story/2230...eason-8-finale
http://www.fimfiction.net/story/2445...s-to-equestria
http://www.fimfiction.net/story/2164...questria-girls
=TtH=
{Works}
http://www.tthfanfic.org/Story-24444...p+Gone+Huh.htm

http://www.tthfanfic.org/Story-27681...controlled.htm
http://www.tthfanfic.org/Story-30468...+in+Heaven.htm
http://www.tthfanfic.org/Story-24355...ge+of+Pace.htm

[thatblondelady]

I get an error message while updating my anthologies. This happens on both AO3 and FFN. The story updates properly, but I get the following error message about not being able to update metadata.

Spoiler:

calibre, version 2.20.0
ERROR: Error Updating Metadata: <p>An error has occurred while FFDL was updating calibre's metadata for <a href=''>Abducted</a>.</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1267, in update_books_loop
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1679, in update_metadata
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ util", line 35, in get_ffdl_config
IOError: [Errno 32] Broken pipe

Story Details:begin:
None
series:
None
end:
None
author_sort:
[u'kenyancougar']
timestamp:
2015-03-12 22:04:43.292520-07:00
url:

title:
Abducted
anthology_meta_list:
category:
True
author:
True
extratags:
True
characters:
True
rating:
True
storyId:
True
follows:
True
reviews:
True
authorHTML:
True
language:
True
site:
True
authorUrl:
True
favs:
True
version:
True
authorId:
True
siteabbrev:
True
genre:
True
lastupdate:
True
langcode:
True
storyUrl:
True
author:
[u'kenyancougar']
comments:

publisher:
www.fanfiction.net
all_metadata:
datePublished:
2014-04-23 13:45:46
rating:
T, T, T
langcode:
en, en, en
version:
FanFictionDownLoader v2.1.4, FanFictionDownLoader v2.1.4, FanFictionDownLoader v2.1.4
formatname:

dateUpdated:
2015-03-11 15:26:38
numChapters:
128
anthology:
true
title:
Abducted
reviews:
236, 519, 409
language:
English, English, English
site:
www.fanfiction.net, www.fanfiction.net, www.fanfiction.net
seriesHTML:

warnings:

follows:
174, 261, 303
characters:
Felicity S., John D., Oliver Q., Walter S., Felicity S., John D., Oliver Q., Sara L., Felicity S., OC, Oliver Q., Roy H.
numWords:
278231
seriesUrl:

authorHTML:
<a class='authorlink' href='https://www.fanfiction.net/u/5570277/kenyancougar'>kenyancougar</a>, <a class='authorlink' href='https://www.fanfiction.net/u/5570277/kenyancougar'>kenyancougar</a>, <a class='authorlink' href='https://www.fanfiction.net/u/5570277/kenyancougar'>kenyancougar</a>
favs:
199, 178, 167
description:
People close to Oliver start to go missing. How will Oliver find them, and what will happen when he does? A story about a true hero! Warning: There are multiple mentions of torture so please be aware as you read.

This is the sequel to Abducted. (I strongly suggest reading that first if you have not already). An old acquaintance of Sara and Oliver has come to town and is looking for revenge. Will Oliver, Team Arrow and the rest of the group be safe? (Does contain Olicity!) Also contains mentions of torture so enter at your own risk. *Now Complete*

Oliver proposes, Roy faces someone from his past, the Team deals with a new threat in town, and a Happily Ever After is within reach! Features all of Team Arrow, the Queen and Lance families, and Sin. This is the final installment in my 'Abducted' series. (Reading Abducted and Teamwork are not necessary, just highly recommended.) *Slight Season 3 Premier Spoilers*
siteabbrev:
ffnet, ffnet, ffnet
genre:
Hurt-Comfort, Friendship, Hurt-Comfort, Family, Romance
category:
Arrow, Arrow, Arrow
storyUrl:
https://www.fanfiction.net/s/10293953/1/, https://www.fanfiction.net/s/10379655/1/, https://www.fanfiction.net/s/10725946/1/
formatext:

ships:

dateCreated:
2015-03-12 22:04:43
storyId:
10293953, 10379655, 10725946
author:
kenyancougar
extratags:
FanFiction, FanFiction, FanFiction
authorId:
5570277, 5570277, 5570277
output_css:

lastupdate:
Last Update Year/Month: 2014/05, Last Update: 2014/05/26, Last Update Year/Month: 2014/10, Last Update: 2014/10/06, Last Update Year/Month: 2015/03, Last Update: 2015/03/11
authorUrl:
https://www.fanfiction.net/u/5570277/kenyancougar, https://www.fanfiction.net/u/5570277/kenyancougar, https://www.fanfiction.net/u/5570277/kenyancougar
site:

calibre_id:
1023
good:
True
comment:

updatedate:
2015-03-11 15:26:38-07:00
pubdate:
2014-04-23 13:45:46-07:00
outfile:
/var/folders/g3/wpr7tn5n6zb9rrdhqcqhlcbw0000gn/T/calibre_2.20.0_tmp_u4aXcu/ffdl_anthology_NA4q1i/fpknce.epub
added:
False
tags:
[u'FanFiction', u'John D.', u'Hurt-Comfort', u'Felicity S.', u'Walter S.', u'Oliver Q.', u'Arrow', u'Sara L.', u'Friendship', u'Roy H.', u'OC', u'Family', u'Romance', 'Anthology']

It's not really that big of a deal to me, because I can just click OK and move on with my life, but I thought I would mention it, in case it was happening to other people.

[MGSaintz]

Doesn't work for me:
http://www.tthfanfic.org/Story-30909...into+a+bar.htm
In fact any story I try to load onto Calibre from tthfanfic gives me the error.

[cryzed]

These stories all work for me, and all the errors definitely point to an unstable internet connection; especially things like "IOError: [Errno 32] Broken pipe" (thatblondelady). Is it possible that you guys are all using the same ISP and it's just failing majorly currently? It's a strange coincidence that so many people would suddenly be faced with the exact same problem (or maybe it's just that all people with this problem suddenly start posting, seeing that someone else did?), but I can't at all reproduce the problem and neither can Jimm seemingly. As far as I know there have been no changes to FFDL recently that reach that deep into its internals, and as previously said these errors can't be fixed by us, they all trace back to a broken socket. Do you ever get any strange error when loading the pages manually in your browser? Such as an incomplete website appearing, or just some kind of error page?

@Jimm sgzmd seems to have already gone ahead and migrated the repository to GitHub... that was a bit fast for my liking... it seems we'll have to become familiar with Git very quickly now:

please stop using Google Code as the changes will not get replicated. I will update the page.

[JimmXinu]

Originally Posted by thatblondelady

I get an error message while updating my anthologies. This happens on both AO3 and FFN. The story updates properly, but I get the following error message about not being able to update metadata.

calibre, version 2.20.0
ERROR: Error Updating Metadata: <p>An error has occurred while FFDL was updating calibre's metadata for <a href=''>Abducted</a>.</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 1267, in update_books_loop
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 1679, in update_metadata
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_util", line 35, in get_ffdl_config
IOError: [Errno 32] Broken pipe

It's not really that big of a deal to me, because I can just click OK and move on with my life, but I thought I would mention it, in case it was happening to other people.

Given where this error is occurring, I don't think it's to do with network.

On some platforms, calibre is remarkably fussy about print statements in the code that runs in the background job and that line number is for a debug print when a certain exception is caught.

The attached version changes the print to a proper logging output instead. I don't see these problems myself so I can't promise this will fix it, but I think it will.

[cryzed]

That would mean the broken pipe has to do with a messed up stdout of some kind... that didn't occur to me. Let's hope that's it.

[JimmXinu]

Originally Posted by cryzed

@Jimm sgzmd seems to have already gone ahead and migrated the repository to GitHub... that was a bit fast for my liking... it seems we'll have to become familiar with Git very quickly now:

FYI, there's some dev level discussion going on about what to do about Google's announcement that Google Code will be going away. Moving the project source to GitHub is a strong possibility, but we're not jumping just yet after all.

We will try to insulate our users from these issues and changes, but you may see some of the discussion going on.

One item I would be interested in hearing public opinion about is changing the project name to something more unique.

When you Google FanFictionDownLoader there are 3 or 4 different projects with that name that come up. And more than once we've gotten bug reports from people using one of the other programs.

[cryzed]

I remember thinking that "Lemon" was a really smart name , as you might recall this was the name of my project that eventually led me to discover FFDL. But yes, I don't think (re-)using that name would be a very good idea for various reasons... unfortunately I am not very creative by nature. I hope someone else in the community can come up with something neat.