[GUI Plugin] FanFictionDownLoader

[SallyK]

I'm not quite sure whether this is a Calibre problem, or specifically a FFDL one, so I'm sorry for bothering you if there's nothing to be done about it.

I'm running Linux with an Xfce desktop, and since the upgrade to Calibre 2, I've been having problems with the dialog box that you enter story URLs into. It works fine the first time I bring it up, and I can add stories as normal. If I bring it up again, though, it doesn't draw correctly on the screen. You get an unholy mess of the window chrome and Calibre library underneath, and if you try and move the window at all, the mess is permanent until you log out - even changing the wallpaper doesn't redraw it.

I have Ubuntu in a vm, and the same thing doesn't happen on Unity, so it's presumably the interaction between FFDL, Qt 5 and Xfce, but at the moment the FFDL URL box is the only dialog that does this - even the FFDL options dialog is fine. Is there anything that can be done about it, or is it purely an upstream problem?

[JimmXinu]

@SallyK,

I am far from an expert in Qt, but I'll try to explain what I understand.

Most of the dialogs used in FFDL are created, displayed, then destroyed. The Add from URL dialog is created once and then shown, hidden and shown again as needed. That was the pattern I found (and copied) to allow the dialog to be open, but not modal; ie, you can use the main calibre GUI while the Add dialog is up to see if you already have a book or whatever. Unfortunately, I don't remember where I copied it from now.

Nothing FFDL does--even if it was wrong--should be able to do some of the things you're describing. And there are a number of known bugs with Qt5.

So I am going to point at the combination of Qt5 and Xfce. If it happens for other X implementations, I might look into doing a workaround of some kind.

[JimmXinu]

Attached is a new test version of FFDL with some significant under-the-covers changes.

The biggest two changes are:

1 - FFDL now caches retrieved files between stories and between foreground and background processing in the same job.

This means if you're downloading several stories from the same author or even the same category(fandom) on the same site, it should speed up downloads a good bit. Currently, this is only used with adastrafanfic,com fanfiction.net, tenhawkpresents.com and tthfanfic.org.

2 - A new developer, 'John Doe' has contributed code for reading eFiction site in a more generic way. It isn't as complete (no series number info, for example), but might allow for eFiction based sites to be added more easily. So far it's only used for themaplebookshelf.com and fannation.shades-of-moonlight.com.

Let me know what you think.

[SallyK]

Originally Posted by JimmXinu

@SallyK,

I am far from an expert in Qt, but I'll try to explain what I understand.

Most of the dialogs used in FFDL are created, displayed, then destroyed. The Add from URL dialog is created once and then shown, hidden and shown again as needed. That was the pattern I found (and copied) to allow the dialog to be open, but not modal; ie, you can use the main calibre GUI while the Add dialog is up to see if you already have a book or whatever. Unfortunately, I don't remember where I copied it from now.

Nothing FFDL does--even if it was wrong--should be able to do some of the things you're describing. And there are a number of known bugs with Qt5.

So I am going to point at the combination of Qt5 and Xfce. If it happens for other X implementations, I might look into doing a workaround of some kind.

Thank you for the prompt response - I feared as much, but I guess Qt 5 is pretty new, and hopefully a bug fix further upstream will solve the problem sooner or later.

[BookJunkieLI]

Any idea what

u'outfile'

in the comment field of the FFDL log means?

Because the electrical system in my house is seriously crappy a breaker got flipped. My idiot landlord didn't bother to tell me he was going to reset ALL of them so the power to my computer got shut off somewhere between the update calibre and count pages process of a download. I ran a Update Metadata Only to see if I could spot what did and didn't get updated. Most had the Download Epub Completed but 7 had the error message from above.

Any ideas?

[JimmXinu]

Originally Posted by BookJunkieLI

Any idea what

u'outfile'

in the comment field of the FFDL log means?

Not really. If it's continuing to happen and you can gather a debug log of it I'll be able to tell you more.

[BookJunkieLI]

Originally Posted by JimmXinu

Not really. If it's continuing to happen and you can gather a debug log of it I'll be able to tell you more.

Okay. I'll trying running it again in debug mode when I get home tonight.

ETA: I tried running them all again while in debug mode and everything went through fine. If it happens again I'll let you know, hopefully with a debug log. Thanks.

[ILB]

If I try to use "Get Story URLs to Download from Web Page" I get the following error message:
NameError: global name 'self' is not defined

Details:

Spoiler:

calibre, version 2.1.0
FEHLER: Nicht abgefangener Fehler: <b>NameError</b>:global name 'self' is not defined

calibre 2.1 isfrozen: True is64bit: False
Windows-Vista-6.0.6002-SP2 Windows ('32bit', 'WindowsPE')
('Windows', 'Vista', '6.0.6002')
Python 2.7.8
Windows: ('Vista', '6.0.6002', 'SP2', 'Multiprocessor Free')
Successfully initialized third party plugins: FanFictionDownLoader && EpubMerge && Count Pages && Kindle Collections && Similar Stories && Quick Preferences && Copy Cover To Device && Reading List && Find Duplicates && Manage Series
Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 403, in get_urls_from_page_menu
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 419, in get_urls_from_page
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.geturls", line 37, in get_urls_from_page
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.adapters.__init__", line 192, in getAdapter
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.adapters.adapter_archiveofourownorg", line 82, in getSiteExampleURLs
NameError: global name 'self' is not defined

for example: http://archiveofourown.org/users/xDa...kDesires/works

What am I doing wrong or where do I have a wrong setting?

[JimmXinu]

Originally Posted by ILB

If I try to use "Get Story URLs to Download from Web Page" I get the following error message:
NameError: global name 'self' is not defined
...
What am I doing wrong or where do I have a wrong setting?

You didn't do anything wrong--this one's my mistake. But then, that's why we do test versions.

A fixed version is attached. Thanks for finding and reporting this problem.

UPDATE Newer version already, see a couple posts down.

[ILB]

Originally Posted by JimmXinu

You didn't do anything wrong--this one's my mistake. But then, that's why we do test versions.

A fixed version is attached. Thanks for finding and reporting this problem.

Thanks for the fast fix!

[JimmXinu]

Another test version, this one extends the new page caching feature to the adapters for AO3, FimFic, portkey and buffynfaith.net.

Again, this new feature should speed up downloads for some sites, most notably ffnet.

I'd appreciate hearing how it's working for other people. Thanks.

[Lzyslckr]

u'cookiejarfile' is what shows up in the comment box. The reason I'm trying it is because the download loads forever. Had a one story download go on for 180 minutes.

[JimmXinu]

Originally Posted by Lzyslckr

u'cookiejarfile' is what shows up in the comment box. The reason I'm trying it is because the download loads forever. Had a one story download go on for 180 minutes.

That's not good. First, I need the story URL so I can try to reproduce the problem.

It would also be very helpful if you could turn on debug output--run calibre from command line as:

calibre-debug -g

...and try doing an FFDL 'Update Always' on that book. Then post the debug output.

[JimmXinu]

Originally Posted by JimmXinu

It would also be very helpful if you could turn on debug output--run calibre from command line as:...
Then post the debug output.

You can also more easily run debug mode from the Preferences drop down or by pressing ctrl-shft-R.

[djrm]

Hello.

I just tried to update an existing book from Near Her Always and I got the following message:

'NoneType' object has no attribute '__getitem__'

for the following story:

http://nha.magical-worlds.us/viewstory.php?sid=6886

Has anyone else gotten this error message for this site?

Thanks.
DJRM