[GUI Plugin] FanFictionDownLoader

[JimmXinu]

Originally Posted by firefoxxy

...Another problem occured today concerning AO3. I tried to download a series but got an error. Can you help?
...
Addition: It seems I broke something in my ini-file when playing with my tagging... Now I cannot download any story from AO3 (same error)
...

That's definitely an error in one of the regular expressions you've entered. The problem is figuring out which one.

Attached is (yet another) test version that adds better error reporting for regular expressions in personal.ini. It will give the error, what regex it occurred in, and what line that regex is part of.

[cyrus_775]

Originally Posted by JimmXinu

Attached test version contains previous changes plus changes for transition from thewriterscoffeeshop.com to twcslibrary.net.

I have been away and during that time, twcs did the switching of the servers, and obviously, is not registering with the fanfiction downloader. But this test version was also removed as well. Will a new update be released soon to the plugin then?

EDIT: in the time I was writing this, the post above came through. Nevermind.

[baggins41]

Originally Posted by JimmXinu

There are tons of settings in FFDL. The three I've highlighted below should all be on to update the metadata cover, as I recall.

Thanks, Jimm! "Update Calibre Cover when Updating Metadata?" was the piece I was missing. Seems obvious in hindsight, but I think I was expecting that setting to control whether or not the "Generate Cover" plugin was run, rather than updating the metadata with the existing cover (even though that's exactly what it said on the tin).

[JimmXinu]

Version 2.0.9 - 03 Nov 2014

• Fix for changed login behavior for the fanfiktion.de adapter (Thanks, cryzed).
• Changes for potterfics.com(Spanish) to login for adult rated stories.
• Updates for Fimfiction.net changes, fixes for image inclusion and password protected stories (Thanks Facedeer and cryzed).
• Change thewriterscoffeeshop.com to twcslibrary.net. (you'll need to change your personal.ini from [www.thewriterscoffeeshop.com] to [www.twcslibrary.net] if you saved your TWCS login.)
• Fix for ficwad.com adult rated stories to properly log in.
• Improved error reporting for regular expressions in personal.ini.

[Tanjamuse]

I don't know what is going on with AO3, but when I try to "Get Story URLs from Web Page" I don't get the ones that is on the page I've chosen.

Which is very weird.

I tried this link, and several others:

http://archiveofourown.org/tags/Harr...orks?page=3275

but they all show other stories than there are on the page

[JimmXinu]

Originally Posted by Tanjamuse

I don't know what is going on with AO3, but when I try to "Get Story URLs from Web Page" I don't get the ones that is on the page I've chosen.

The page cache scheme introduced a few versions back was interfering with the special login code for 'get from web page' with AO3.

The attached test version should correct that issue.

[skb]

Hi

Just upgraded to new mac (yay) and updated Calibre (double yay) but have had problems importing stories from AO3.

Get this error (on all story attempts so far)

Spoiler:

Status
Title
Author
Comment
URL
Bad
Unknown
Unknown
<urlopen error [Errno 4] Interrupted system call>
http://archiveofourown.org/works/315576

The only thing that has changed on Mac/Calibre (apart from upgrading) is that I tried to install the plugin(s) to upload stories for Marvin reader (didn't work). ffnet seems to still work fine (irony) but AO3 not so much.

Mac 10.10
Calibre 2.8
FFDL 2.09

Edited: I just tried the "download from webpage" feature (which I'm embarrassed to admit I didn't know about) and it mostly worked. It downloaded most of the stories (including the one above that wouldn't work) but it threw the same error on another and a "Connection reset by peer" on two.

Spoiler:

Status
Title
Author
Comment
URL
Add
A Missing Piece
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315023
Add
Oasis
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/237036
Add
Secrets and Truths
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/316020
Add
Portents and Presents
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315677
Add
Hard Lessons Learned
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/316027
Add
No Holds Barred
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/316016
Add
Babylon
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315576
Add
Seasons
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315016
Add
Stolen Moments
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315821
Add
Call to Arms
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315767
Add
Out of Sight, Out of Mind, Into Heart
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315050
Add
S is for Silence
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315010
Add
Mail Call
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315836
Add
Righting the Wrongs
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315782
Add
Fusion
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315979
Add
A Dream Fulfilled
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315584
Add
Mending Bridges, Healing Souls
Ralkana
Download epub completed, 1 chapters.
http://archiveofourown.org/works/315927
Bad
Unknown
Unknown
<urlopen error [Errno 4] Interrupted system call>
http://archiveofourown.org/works/315000
Bad
Unknown
Unknown
[Errno 54] Connection reset by peer
http://archiveofourown.org/works/315037
Bad
Unknown
Unknown
[Errno 54] Connection reset by peer
http://archiveofourown.org/works/316509

[JimmXinu]

Originally Posted by skb

Just upgraded to new mac (yay) and updated Calibre (double yay) but have had problems importing stories from AO3.
...

<urlopen error [Errno 4] Interrupted system call>

This error indicates something happened while reading or waiting for network traffic.

[Errno 54] Connection reset by peer

This error indicates that the connection was closed by the far end.

I suspect that there was a network issue somewhere between you and AO3. Or possibly that you have firewall settings that are causing you problems. I'm able to download all the URLs you posted without difficulty.

[Saioko]

Hi.
It seems that http://ficbook.net made some changes after they were under attack over weekend. I tried to download some stories, but they all returned an error "HTTP Error 403: Forbidden"
Here is an url example: http://ficbook.net/readfic/1500001

[JimmXinu]

Originally Posted by Saioko

It seems that http://ficbook.net made some changes after they were under attack over weekend. I tried to download some stories, but they all returned an error "HTTP Error 403: Forbidden"
Here is an url example: http://ficbook.net/readfic/1500001

The page caching I added a couple versions back wasn't honoring the User-Agent set from configuration.

Try this version, it's working for me.

[skb]

Originally Posted by JimmXinu

<urlopen error [Errno 4] Interrupted system call>

This error indicates something happened while reading or waiting for network traffic.

[Errno 54] Connection reset by peer

This error indicates that the connection was closed by the far end.

I suspect that there was a network issue somewhere between you and AO3. Or possibly that you have firewall settings that are causing you problems. I'm able to download all the URLs you posted without difficulty.

Thanks for checking Jim. I'll have another try at downloading.
cheers,

[Saioko]

Originally Posted by JimmXinu

The page caching I added a couple versions back wasn't honoring the User-Agent set from configuration.
Try this version, it's working for me.

Thank you. This one works.

[Saioko]

Hi again.
Second question. 'Get IRLs from web-page' now doesn't work with fanfiction.net
Here's what I get when I try to get url's from this page https://www.fanfiction.net/u/3975347/celticsouthern:

Spoiler:

calibre, version 2.8.0
ОШИБКА: Необработанное прерывание:: <b>TypeError</b>:_fetchUrl() got an unexpected keyword argument 'usecache'

calibre 2.8*  isfrozen: True is64bit: False
Linux-3.13.0-24-generic-i686-with-debian-jessie-sid Linux ('32bit', 'ELF')
('Linux', '3.13.0-24-generic', '#46-Ubuntu SMP Thu Apr 10 19:08:14 UTC 2014')
Python 2.7.8
Linux: ('debian', 'jessie/sid', '')
Successfully initialized third party plugins: Generate Cover && FanFictionDownLoader
Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 403, in get_urls_from_page_menu
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 419, in get_urls_from_page
  File "calibre_plugins.fanfictiondownloader_plugin.fanficdownloader.geturls", line 60, in get_urls_from_page
TypeError: _fetchUrl() got an unexpected keyword argument 'usecache'

[JimmXinu]

Originally Posted by Saioko

Hi again.
Second question. 'Get IRLs from web-page' now doesn't work with fanfiction.net
Here's what I get when I try to get url's from this page https://www.fanfiction.net/u/3975347/celticsouthern:

This is why we do test versions. Here's another fix.

[Saioko]

Originally Posted by JimmXinu

This is why we do test versions. Here's another fix.

Thank you!
Wow! It is even fast now! Previous version could take up to 10 minutes to get one story from fanfiction.net, this one took about 10 to get whole 50 stories.

Oh, not a big problem, but I wanted to ask. Right now it grabs links to all stories - both by author and in Favorite stories. A while ago (a few months at least) it only got stories by author. Was it a deliberate change or something was changed on fanficton.net and it is now impossible to separate those two blocks of links?