[GUI Plugin] FanFictionDownLoader

[JimmXinu]

Originally Posted by BookJunkieLI

Woohoo! I added that little bit of code to the personal.ini then setup 3 test columns - text, shown in tag browser; comma separated text, contains names; and Long text, like comments; then directed Relationships to populate those 3 columns. I ran a Metadata Update Only and all three columns populated with the correct data in the correct order. I even restarted Calibre and it's still in the proper order.

I expected that the text & long text would preserve the order, but I'm surprised about comma separated text. I guess the 'contains names' checkbox does more than I thought.

I'm glad it's working for you.

[JimmXinu]

Originally Posted by BookJunkieLI

Out of curiosity, can that bit of code - keep_in_order_ships:true - be used for any other archive that lists multiple pairings or is it specific to AO3? I really only use AO3, with occasional forays over to FFN.Net, so I don't what other archives list multiple pairings but, if others do, can it be used for them as well?

You can use keep_in_order_*:true for any metadata key for any site.

Some examples that would effect all sites (because of the [defaults]):

[defaults]
keep_in_order_ships:true
keep_in_order_characters:true
keep_in_order_genres:true

And ffnet stories can have multiple pairings.

[Firedancer885]

Jimm,

please check this story: http://archiveofourown.org/works/1767139
The series page list one number for the story's place in the series, the story page has a different one.

[cryzed]

Honestly, that seems like an A03 bug, you'd better report this issue to them. If we changed the parsing now it's likely to break in the future, unless we parse the series overview page which would add unnecessary overhead to request times etc. Try reporting the bug to them first please.

[Firedancer885]

Ok, thanks

[cryzed]

If they don't fix it in the coming weeks please mention it again and I'm sure we can do something about it, it's just that fixing other people's mistakes (specifically AO3) in the code base is just not a very good idea and has a few disadvantages, most prominently it's prone to change or to get fixed and then the whole code handling their mistake either breaks or does unnecessary things that only slow things down unnecessarily. Sorry about this. If it's just me and Jimm is of a completely different opinion he'll mention it hopefully.

[BookJunkieLI]

Originally Posted by JimmXinu

I expected that the text & long text would preserve the order, but I'm surprised about comma separated text. I guess the 'contains names' checkbox does more than I thought.

Yep. Kovid had suggested using that option, but without your little bit of code Calibre was pulling the tags in randomly so it didn't quite do what I wanted. Apparently there is a small glitch in there, it's discussed in my original thread about the problem https://www.mobileread.com/forums/sho...=240844&page=2 in posts #16 and #17. Something to do with the metadata editor. Chaley is planning to put through a fix, likely in the next update.

[BookJunkieLI]

Originally Posted by JimmXinu

You can use keep_in_order_*:true for any metadata key for any site.

Some examples that would effect all sites (because of the [defaults]):

[defaults]
keep_in_order_ships:true
keep_in_order_characters:true
keep_in_order_genres:true

And ffnet stories can have multiple pairings.

Awesome.

[JimmXinu]

Originally Posted by Firedancer885

please check this story: http://archiveofourown.org/works/1767139
The series page list one number for the story's place in the series, the story page has a different one.

I'm seeing it as 29 both on the story page and the series page.

So I suspect either they've fixed it, or there's a latency issue somewhere that can cause parts of their system to be out of sync.

If it crops up a lot, we'll look at it in more depth. But I agree with cryzed's first reaction--the site being inconsistent shouldn't be our problem.

[JimmXinu]

Here's a test version that again has all of cryzed's committed changes and a change to the metadata splitting feature to make it safer. You shouldn't be able to tell the difference--unless you had a story with \, in its metadata from the author.

I'd appreciate it if Amalthia or one of the other metadata split users would give it a try and let me know.

I hope to put out an official new release today or tomorrow.

[honeysyrup]

Hi, I'm having trouble downloading this fic:
https://www.fanfiction.net/s/5337624...ce-from-A-to-B

I get this error message:

Spoiler:

FFDL:DEBUG:calibre_plugins.fanfictiondownloader_pl ugin.fanficdownloader.adapters
.adapter_fanfictionnet(90):URL: https://www.fanfiction.net/s/5337624/1/
FFDL:ERROR:calibre_plugins.fanfictiondownloader_pl ugin.dialogs(600):Exception: {
u'comment': u'expected string or buffer', u'publisher': u'www.fanfiction.net', u
'begin': None, u'good': False, u'end': None, u'author': [u'zero-damage'], u'url'
: u'https://www.fanfiction.net/s/5337624/1/', u'listorder': 0, u'author_sort': [
u'zero-damage'], u'site': 'www.fanfiction.net', u'comments': u'', u'calibre_id':
None, u'added': False, u'all_metadata': {'rating': u'T', 'author': u'zero-damag
e', 'series': '', 'numWords': u'309,117', 'site': u'www.fanfiction.net', 'datePu
blished': u'2009-08-28 07:50:56', 'authorHTML': u"<a class='authorlink' href='ht
tps://www.fanfiction.net/u/1861271/zero-damage'>zero-damage</a>", 'storyUrl': u'
https://www.fanfiction.net/s/5337624/1/', 'storyId': u'5337624', 'category': u'P
ersona Series', 'lastupdate': u'Last Update Year/Month: 2013/08, Last Update: 20
13/08/01', 'formatext': '', 'title': u'The Shortest Distance from A to B', 'ship
s': '', 'seriesHTML': '', 'formatname': '', 'version': u'FanFictionDownLoader v1
.8.23', 'dateUpdated': u'2013-08-01 18:26:02', 'numChapters': u'45', 'status': u
'In-Progress', 'langcode': u'en', 'description': u"Persona 4. A record of Kanji
Tatsumi's year - specifically, his interactions with and raging crush on one dif
ficult pint-size detective.", 'warnings': '', 'dateCreated': u'2014-06-12 18:51:
55', 'output_css': '', 'authorId': u'1861271', 'authorUrl': u'https://www.fanfic
tion.net/u/1861271/zero-damage', 'characters': u'Kanji T., Naoto S.', 'genre': u
'Drama, Humor', 'favs': u'405', 'language': u'English', 'follows': u'394', 'seri
esUrl': '', 'reviews': u'625', 'siteabbrev': u'ffnet', 'extratags': u'FanFiction
'}, u'title': u'The Shortest Distance from A to B'}:expected string or buffer
Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.dialo gs", line 590, in do_lo
op
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 896, in p
rep_download_loop
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.story", lin
e 625, in getSubjectTags
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.configurabl
e", line 184, in getConfigList
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.configurabl
e", line 157, in getConfigList
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.configurabl
e", line 150, in get_config_list
File "re.py", line 167, in split
TypeError: expected string or buffer
FFDL:DEBUG:calibre_plugins.fanfictiondownloader_pl ugin.ffdl_plugin(726):FanFicti
onDownLoader v1.8.23
FFDL:DEBUG:calibre_plugins.fanfictiondownloader_pl ugin.ffdl_plugin(761):url:http
s://www.fanfiction.net/s/5337624/1/

[JimmXinu]

Originally Posted by honeysyrup

Hi, I'm having trouble downloading this fic:
https://www.fanfiction.net/s/5337624...ce-from-A-to-B
...

The story downloaded fine for me.

Are you using the official release, or one of the recent test versions?

The error trace suggests a problem in your personal.ini in the include_subject_tags or extra_subject_tags parameters. What do you have in your personal.ini?

[honeysyrup]

Originally Posted by JimmXinu

The story downloaded fine for me.

Are you using the official release, or one of the recent test versions?

The error trace suggests a problem in your personal.ini in the include_subject_tags or extra_subject_tags parameters. What do you have in your personal.ini?

I recently upgraded FFDL from an older version, so I think my personal.ini was out of date. I didn't realize it had problems because it worked fine on other fics. I replaced it with the default ini, and it worked this time. Thanks for your such a quick response! Sorry to bother you.

[JimmXinu]

Version 1.8.24 - 14 Jun 2014

Now supporting over 100 different sites! Thanks, cryzed, for pushing us over the top.

• New site: nocturnal-light.net -- Thanks, cryzed!
• New site: fanfic.hu (Hungarian language) -- Thanks, cryzed!
• New site: fanfiction.csodaidok.hu (Hungarian language) -- Thanks, cryzed!
• Updates for sites: bloodshedverse.com, voracity2.e-fic.com -- Thanks, cryzed!
• Fix for making anthology books when source books don't have number of words.
• New '\,' feature to 'split' metadata items in replace_metadata.

Example:

replace_metadata:
 category=>Captain America \(Movies\)=>Captain America\,Marvel

The category list will then contain both "Captain America" and "Marvel".

[Amalthia]

Originally Posted by JimmXinu

Here's a test version that again has all of cryzed's committed changes and a change to the metadata splitting feature to make it safer. You shouldn't be able to tell the difference--unless you had a story with \, in its metadata from the author.

I'd appreciate it if Amalthia or one of the other metadata split users would give it a try and let me know.

I hope to put out an official new release today or tomorrow.

Hi,

I tested this new change and it appears to work.

However, I'm curious to know what the main difference is between using the one line like this:

#fandoms=>(Captain America|Thor|Avengers|Agents of S.H.I.E.L.D.|X-Men|Spider-Man)=>\1{{,}}Marvel

Or the \, option?

Best Regards,
Amalthia