[GUI Plugin] FanFictionDownLoader

[cryzed]

Jimm, did you consider switching to Beautiful Soup 4? It should be 100% backwards-compatible and have an even more robust parser IIRC.

[JimmXinu]

Originally Posted by cryzed

Jimm, did you consider switching to Beautiful Soup 4? It should be 100% backwards-compatible and have an even more robust parser IIRC.

I believe I gave a try once before and rejected it, but I don't remember why exactly.

I tried again now and ran into problems including it in the plugin. bs4 is designed and packaged to be installed like a good python package--not bundled into the somewhat odd distro zips that calibre plugins use. And I also support CLI and Google AppEngine deployment from the same code.

I may investigate a bit more with CLI, but I don't have high hopes.

[Firedancer885]

Still the same story. Why did I get funny characters instead of quote marks?
http://www.ncisfiction.com/story.php?stid=05113

[JimmXinu]

Originally Posted by Firedancer885

Still the same story. Why did I get funny characters instead of quote marks?
http://www.ncisfiction.com/story.php?stid=05113

The web site says it's material is encoded with iso-8859-1. But that particular story is encoded with UTF8.

You can add this to your personal.ini to fix that story, but you may need to remove it later for other stories:

[www.ncisfiction.net]
website_encodings:utf8, iso-8859-1, Windows-1252

[Firedancer885]

Originally Posted by JimmXinu

The web site says it's material is encoded with iso-8859-1. But that particular story is encoded with UTF8.

You can add this to your personal.ini to fix that story, but you may need to remove it later for other stories:

[www.ncisfiction.net]
website_encodings:utf8, iso-8859-1, Windows-1252

It doesn't work. I tried overwrite alway, tried deleting then readding the story, I still get the weird characters. I even tried changing ncisfiction.net to ncisfiction.com, nada.

[JimmXinu]

Originally Posted by Firedancer885

It doesn't work. I tried overwrite alway, tried deleting then re-adding the story, I still get the weird characters. I even tried changing ncisfiction.net to ncisfiction.com, nada.

In the paragraph:

Spoiler:

“Call me Abby.” She placed her gift bags on the Feline's desk and dug into them. First she pulled out a CD case and something that looked like a book under all that - matching the gift bags �" in metallic pink wrapping paper. Both items got handed over to Todd. Then something small that ended in a blushing McGee's fingers.<br />

There's a character that can't be interpreted as UTF8: gift bags �"
So the UTF8 decoding fails and it falls back to iso-8859-1 which doesn't care as much.

Nothing I can do about it when the author puts bad characters in. Sorry.

But it's a completed one-shot, so just copy-pasting the text in using Sigil or Tweak Book should be easy.

[ILB]

If I download an anthology direct with "For Antholoy EPUB" in comments is a summary of the titles of the books.
Is it possible to change that to the Description on the series-page?
For example: http://archiveofourown.org/series/10723

[Firedancer885]

Originally Posted by JimmXinu

In the paragraph:



There's a character that can't be interpreted as UTF8: gift bags �"
So the UTF8 decoding fails and it falls back to iso-8859-1 which doesn't care as much.

Nothing I can do about it when the author puts bad characters in. Sorry.

But it's a completed one-shot, so just copy-pasting the text in using Sigil or Tweak Book should be easy.

Thanks

[JimmXinu]

Originally Posted by ILB

If I download an anthology direct with "For Antholoy EPUB" in comments is a summary of the titles of the books.
Is it possible to change that to the Description on the series-page?
For example: http://archiveofourown.org/series/10723

Not at this time. The code that gets story URLs for anthologies isn't site specific, so it take some re-architecting to do that.

[JimmXinu]

Version 1.7.44 - 14 Sept 2013

• Remove dateutil dependency
• Fix for certain portkey.org stories.
• Fix for PI for int/float custom columns setting empty.

[ILB]

Originally Posted by JimmXinu

Not at this time. The code that gets story URLs for anthologies isn't site specific, so it take some re-architecting to do that.

OK, but thanks anyway.

[JimmXinu]

Attached is a version with a fix for whofic.com.

UPDATE Sep 25, 2013 - Remove obsolete beta versions

[Sefiriot]

Jimm, or anyone who's got this figured out: I've got some downloaded fics where there were cover images included, but since I've set Generate Cover to run by default on new books in FFDL (so anthologies would also have covers generated automatically), it generates the default cover instead of using the included cover image for those fics. Is there a setting where I can automatically generate the default cover for downloaded fics, but still have it respect make_firstimage_cover:true in personal.ini where there's images in the epub?

[JimmXinu]

Sefiriot -

If it's anthologies you're trying to use Generate Cover for, I use this:

[defaults]
add_to_generate_cover_settings:
 ${anthology}=>true=>FFDL

Where FFDL is the name of the GC settings.

[Amalthia]

Hi,

I'm still receiving the cannot update metadata on ebooks and I think part of the problem is if there is no data to fill in the custom column (like if a story has 0 reviews) I receive the error message that the metadata couldn't be updated. However, the other custom columns filled in information.

Here is the message I recieved on the latest story download.

calibre, version 1.3.0
ERROR: Error Updating Metadata: <p>An error has occurred while FFDL was updating calibre's metadata for <a href='http://archiveofourown.org/works/968232'>The Victory of Shadows</a>.</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1113, in update_books_loop
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1512, in update_metadata
File "site-packages\calibre\db\legacy.py", line 646, in set_custom
File "site-packages\calibre\db\cache.py", line 939, in set_field
File "site-packages\calibre\db\write.py", line 509, in set_books
File "site-packages\calibre\db\write.py", line 509, in <dictcomp>
File "site-packages\calibre\db\write.py", line 88, in adapt_number
ValueError: invalid literal for int() with base 10: ''