[GUI Plugin] FanFictionDownLoader

[shesgottaread]

@FaceDeer, would you mind sharing the regexes needed to do this correcting of the author_sort? I'm trying to learn python, and have done some other programming, but regexes are something that have always given me problems.

[FaceDeer]

Originally Posted by shesgottaread

@FaceDeer, would you mind sharing the regexes needed to do this correcting of the author_sort? I'm trying to learn python, and have done some other programming, but regexes are something that have always given me problems.

This one's about the simplest possible one.

The goal of this regex is to match the entire string that's in the "author" field and output it unmodified into the "author_sort" field. The regex to match the source "author" field is:

(.*)

The dot means "match any character (that's not a line-end)" and the asterix after it modifies it to mean "match any number of this character". So it'll match a string of any length consisting of any characters. The brackets around them tell the regex engine that this group of matched characters is going to be referred to later, so it should keep track of it.

The regex to put in the destination "author_sort" field is:

\1

The backslash-one tells the regex engine "insert the group of characters that you matched with the first bracketed section of the source regex". Fancier regexes could have multiple bracketed sections, for example if you wanted to reverse the order of two chunks of something you could have bracketed sections match the first chunk and the second chunk and then output to a "\2 \1" regex. I suppose in theory one could come up with a fancy regex to parse the author_sort and reverse-engineer it back into its original form, but since we *have* its original form already sitting in the author column that's not necessary.

I'm still in the process of making a backup of my main fanfic archive so I haven't run this "for real" yet, but I did some test runs on a small test archive so I'm 99% sure this is correct. Nevertheless, make a backup of your own before you run this. Always a good idea when dealing with regexes, even ones this simple. They can be arcane.

[shesgottaread]

Thanks very much! I think I understand what this is doing, and you're right, since we've got the source data, no need to try and reverse-engineer the author_sort back.

Originally Posted by FaceDeer

Always a good idea when dealing with regexes, even ones this simple. They can be arcane.

Arcane is definitely the word for it!

[DavidBrookes]

I am having a problem with downloading some stories with Calibre/FanFictionDownloader from PhoenixSong.Net. I am getting the following error:

calibre, version 0.9.44
ERROR: Unhandled exception: <b>AttributeError</b>:'NoneType' object has no attribute 'replace'

calibre 0.9.44 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.4
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.dialo gs", line 568, in do_loop
File "calibre_plugins.fanfictiondownloader_plugin.dialo gs", line 575, in do_when_finished
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1063, in start_download_job
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1900, in escapehtml
AttributeError: 'NoneType' object has no attribute 'replace'

See below for the storeis giving the problem -

http://www.phoenixsong.net/fanfiction/story/5951/
http://www.phoenixsong.net/fanfiction/story/5980/
http://www.phoenixsong.net/fanfiction/story/5979/
http://www.phoenixsong.net/fanfiction/story/5981/

[JimmXinu]

Originally Posted by DavidBrookes

I am having a problem with downloading some stories with Calibre/FanFictionDownloader from PhoenixSong.Net. I am getting the following error:
...
http://www.phoenixsong.net/fanfiction/story/5951/
http://www.phoenixsong.net/fanfiction/story/5980/
http://www.phoenixsong.net/fanfiction/story/5979/
http://www.phoenixsong.net/fanfiction/story/5981/

Okay, I know what the problem is. It's something we've bumped into before when sites allow HTML tags in the title.

Up until now, I've be changing it for one site as a time as it becomes a problem. This time I've gone through and fixed it for all sites. But that ends up touching the adapters for 75 different sites.

So here's a test version for people to use for a bit to make sure I didn't break anything in the process. :-)

[ILB]

I updated to calibre 1.0 and now if I add new stories my custom columns are no longer filled.
I checked again and in my settings in FFDL in custom columns I have definded a few of my columns to set automatically - but that doesn´t happen anymore.
Do I need to adjust something?

[JimmXinu]

@ILB - I'm not seeing this problem.

What kind of column is it that isn't working? Int, float, text?

What is the metadata that you're trying to fill it with?

[El_Pirato]

why is adultfanfiction.net not on your list of sites? Is it because of the electronic signature?

[JimmXinu]

Originally Posted by El_Pirato

why is adultfanfiction.net not on your list of sites? Is it because of the electronic signature?

Exactly. The ominous legal warnings are sufficient to make us leery.

[Snowman]

Originally Posted by ILB

I updated to calibre 1.0 and now if I add new stories my custom columns are no longer filled.
I checked again and in my settings in FFDL in custom columns I have definded a few of my columns to set automatically - but that doesn´t happen anymore.
Do I need to adjust something?

I'm seeing the same thing - in my case it is specifically "word count". The message "queuing for Count Pages plugin" (can't remember the exact text) never appears, and the background task is not initiated. I've seen the issue on adding a new story, and both with a single and a bulk update.

The CP plugin itself works fine, and I can manually force a recount.

van

[JimmXinu]

Originally Posted by Snowman

I'm seeing the same thing - in my case it is specifically "word count". The message "queuing for Count Pages plugin" (can't remember the exact text) never appears, and the background task is not initiated. I've seen the issue on adding a new story, and both with a single and a bulk update.

The CP plugin itself works fine, and I can manually force a recount.

Failure to launch Count Pages is a very different problem from failing to fill custom columns. And not one I'd ever notice--I don't use it regularly.

However, I don't see that problem, either. I configure CP for the column and check 'Page Count' on the Count Pages tab in FFDL config and it works.

What OS are people seeing problems with?

[Snowman]

Originally Posted by JimmXinu

Failure to launch Count Pages is a very different problem from failing to fill custom columns. And not one I'd ever notice--I don't use it regularly.

However, I don't see that problem, either. I configure CP for the column and check 'Page Count' on the Count Pages tab in FFDL config and it works.

What OS are people seeing problems with?

win xp pro sp2 here.

The column set up (from metadata_db_prefs_backup.json) is

Spoiler:

"#wordcount": {
"is_category": false,
"colnum": 9,
"kind": "field",
"is_custom": true,
"name": "Words",
"column": "value",
"rec_index": 24,
"search_terms": [
"#wordcount"
],
"link_column": "value",
"label": "wordcount",
"is_multiple": {},
"datatype": "int",
"category_sort": "value",
"table": "custom_column_9",
"is_editable": true,
"display": {
"number_format": "{:,}"
},
"is_csp": false
},

[ILB]

Originally Posted by JimmXinu

@ILB - I'm not seeing this problem.

What kind of column is it that isn't working? Int, float, text?

What is the metadata that you're trying to fill it with?

I use the following columns and no one of these works:
updated (typ: date)
word count (numbers)
status (text)
relationship (text)
category (text)

What OS are people seeing problems with?

Windows Vista


I agree with snowman, a manually running of count pages works and fills the column word count.

[SuzanneTF]

For a few stories I'm attempting to download, I get the error "'NoneType' object has no attribute 'nextSibling'." Other stories are downloading fine. Is that something wrong with the story that is uploaded on the site?

[JimmXinu]

Originally Posted by SuzanneTF

For a few stories I'm attempting to download, I get the error "'NoneType' object has no attribute 'nextSibling'." Other stories are downloading fine. Is that something wrong with the story that is uploaded on the site?

Please post story URLs exhibiting the problem.