[GUI Plugin] FanFictionDownLoader

[cryzed]

Attached is a fixed version.

EDIT: Regarding the missing special characters -- they don't even display correctly for me in my browser, and Firefox should definitely be able to handle those. I suspect it's an issue with the site. Can you give me a link to a story where it displays those characters correctly on the site but not within the story file?

[theducks]

Originally Posted by matteo65

@cryzed

In the Summary I see some special Hungarian characters are missing and replaced by '?' marks. Maybe a character encoding issue?
The problematic characters are ő,Ő,ű,Ű

The summary as it looks now:
Mi történik, ha négy ?rültet szabadítanak rá a Roxfort mit sem sejt? népére?

And the correct version:
Mi történik, ha négy őrültet szabadítanak rá a Roxfort mit sem sejtő népére?

Character encoding problem or missing glyph in the font being used.
The ? means there is something there, just no available glyph. That could be either encoding or font.

[cryzed]

Here's the relevant URL: http://fanfiction.csodaidok.hu/viewuser.php?uid=1660. I'm pretty sure it's the site's error, seeing as these character are displayed correctly in the chapters' text. It seems to be an actual '?' character. They probably changed their encoding at some point and then handled conversion incorrectly. Only the metadata on the authors' pages seem to be affected.

[matteo65]

@cryzed
Thank you for the fix, now I can download this story,
but the language is still set to English.
Regarding the special characters you are right: it is an issue with the site.

Originally Posted by cryzed

Attached is a fixed version.

EDIT: Regarding the missing special characters -- they don't even display correctly for me in my browser, and Firefox should definitely be able to handle those. I suspect it's an issue with the site. Can you give me a link to a story where it displays those characters correctly on the site but not within the story file?

[cryzed]

Fixed in the attached version, sorry about that.

[JimmXinu]

Originally Posted by cryzed

Jimm, since you are already using "\s" for a "forced space", wouldn't it make sense to simply use "\," instead of "{{,}}"? Just a suggestion.

Hmmm. On the one hand, you have a point, it would be more consistent. And '{{,}}' is by no means pretty.

On the other hand, I feel like '\,' is something that people would be more likely to use accidentally or without understanding what they're doing--especially with '\( \)' appearing in the regexp.

I also figure it's slightly more likely to appear by typo in the data.

Anybody else care to register an opinion? I'm tempted to just flip a coin.

[davidfor]

Originally Posted by JimmXinu

Hmmm. On the one hand, you have a point, it would be more consistent. And '{{,}}' is by no means pretty.

On the other hand, I feel like '\,' is something that people would be more likely to use accidentally or without understanding what they're doing--especially with '\( \)' appearing in the regexp.

I also figure it's slightly more likely to appear by typo in the data.

Anybody else care to register an opinion? I'm tempted to just flip a coin.

Are you sure you can't just use a comma? A tag can't actually have a comma. Or at least I can't work out how to enter one in the calibre GUI. The two places I can think of both use the comma as a separator. If you put a comma in the tag via FFDL, it get changed to a semi-colon somewhere along the way. I know I looked at this before, and I think it was where FFDL handed the string the calibre to make a tag.

[JimmXinu]

Originally Posted by davidfor

Are you sure you can't just use a comma? A tag can't actually have a comma. ...

Pretty sure. There's a lot more different types of metadata than just calibre tags. FFDL even lets you configure what the list separator is for the different metadata types.

[BookJunkieLI]

Originally Posted by JimmXinu

Pretty sure. There's a lot more different types of metadata than just calibre tags. FFDL even lets you configure what the list separator is for the different metadata types.

Can I ask how to do that? I've been attempting to get the relationship tags from AO3 to import in the order that they appear on the site. I don't know that this is possible but I've been experimenting with various custom columns. Seeing this, I'm wondering if I change the list separator from a comma to an ampersand and then import either into a comments column or an author style column if it will keep the proper order.

[Tanjamuse]

I've got a story that won't download:


Status
Title
Author
Comment
URL
Bad
Unknown
Unknown
'NoneType' object has no attribute 'string'
http://spikeluver.com/SpuffyRealm/vi....php?sid=28920

[cryzed]

Fixed in the attached version.

[JimmXinu]

Originally Posted by BookJunkieLI

Can I ask how to do that? I've been attempting to get the relationship tags from AO3 to import in the order that they appear on the site. I don't know that this is possible but I've been experimenting with various custom columns. Seeing this, I'm wondering if I change the list separator from a comma to an ampersand and then import either into a comments column or an author style column if it will keep the proper order.

There's two different mechanisms working against you here.

First, FFDL sorts most of the metadata lists itself. But you can disable that for specific metadata lists:

[archiveofourown.org]
keep_in_order_ships:true

Second, calibre sorts all comma separated, tag-like columns whether you want it to or not. So you can only preserve the order if you use a basic text column (the top one on the list). You'll still be able to view and search on that column, but it will be a single string and not show up as separate tags in tag browser, for example.

[JimmXinu]

Here's a test version with all of cryzed's committed changes, plus the replace_metadata split feature, but now splitting on '\,' instead of '{{,}}'.

[BookJunkieLI]

Woohoo! I added that little bit of code to the personal.ini then setup 3 test columns - text, shown in tag browser; comma separated text, contains names; and Long text, like comments; then directed Relationships to populate those 3 columns. I ran a Metadata Update Only and all three columns populated with the correct data in the correct order. I even restarted Calibre and it's still in the proper order.

This seriously just saved me days, possibly weeks, of work.

:

[BookJunkieLI]

Out of curiosity, can that bit of code - keep_in_order_ships:true - be used for any other archive that lists multiple pairings or is it specific to AO3? I really only use AO3, with occasional forays over to FFN.Net, so I don't what other archives list multiple pairings but, if others do, can it be used for them as well?