[GUI Plugin] FanFictionDownLoader

[JimmXinu]

@Jade Aislin -

It's custom_columns_settings (note columnS), not custom_column_settings. That's your first problem.

Re: invalid group reference -- \1 matches the first () group. No (), no group. Needs to be more like:

ships=>(Bilbo Baggins)=>Book/Comics.LOTR.\1

[Jade Aislin]

Originally Posted by JimmXinu

@Jade Aislin -

It's custom_columns_settings (note columnS), not custom_column_settings. That's your first problem.

Re: invalid group reference -- \1 matches the first () group. No (), no group. Needs to be more like:

ships=>(Bilbo Baggins)=>Book/Comics.LOTR.\1

Both changes worked. I had tried using the parenthesis earlier, but I guess it didn't matter because I was using the wrong keyword. Also, the extra metadata is now working and I can figure out how I want to funnel that data.

Thanks.

[Jade Aislin]

I've run into another problem with the ships tags. Some of the ships put the names together with an '&' sign. However, I want '/' between names so that I don't duplicate pairings. For example, I want to only see Benton Fraser/Ray Kowalski, not Benton Fraser & Ray Kowalski.

I first tried:

ships=> & =>/

However, instead of the '/' that I wanted, I got ' /amp; '. Then I added the line:

ships=>^ /amp; $=>/

I even tried to convert the '&' without looking for the spaces around it as well and I tried using ^ & $ to try to get the entire section I wanted to change. It always left the ' /amp; '. Any help would be appreciated.

[JimmXinu]

Because the text comes from an HTML source (and three of the four output formats use HTML), the metadata is kept in memory with HTML entities in it.

So this should work for you:

ships=> & =>/

[Jade Aislin]

Originally Posted by JimmXinu

Because the text comes from an HTML source (and three of the four output formats use HTML), the metadata is kept in memory with HTML entities in it.

So this should work for you:

ships=> & =>/

I didn't realize that the HTML entities were saved. When I put in your code it did change the '&'. However, it left the space before and after.

Would using codes like the '&' work on the spaces? Is there a cheat sheet that would show me the special symbols that I might have trouble replacing?

[JimmXinu]

Originally Posted by Jade Aislin

I didn't realize that the HTML entities were saved. When I put in your code it did change the '&'. However, it left the space before and after.

Hmmm. FFDL may be somewhat overzealous with stripping leading/trailing whitespace from config items. I'll look into it.

In the meantime, you can use this:

replace_metadata:
 ships=>[ ]&[ ]=>/

Originally Posted by Jade Aislin

Would using codes like the '&' work on the spaces? Is there a cheat sheet that would show me the special symbols that I might have trouble replacing?

Not on the spaces, no. The only HTML entities that should remain are:

• &
• <
• >

Everything else should have been converted to unicode chars by that point.

[JimmXinu]

Attached is a version that doesn't strip spaces from around the replace_metadata regular expressions. However, I can't stop the .ini parser stripping trailing spaces. So I've added a '\s'->' ' feature for the replacement string.

## So this works now:
replace_metadata:
 ships=> & =>/

## But to do the reverse, replace name/name with name & name,
## you have to do this:
replace_metadata:
 ships=>/=>\s&\s

UPDATE Apr 27 2013 - Remove obsolete beta versions

[Jade Aislin]

Originally Posted by JimmXinu

Hmmm. FFDL may be somewhat overzealous with stripping leading/trailing whitespace from config items. I'll look into it.

In the meantime, you can use this:

replace_metadata:
 ships=>[ ]&[ ]=>/

It seems to happen rarely, so using the [ ] may be enough. Your code did work for me. I was also able to generalize another tag with them. I wanted to generalize the character: Ori - Character, by creating a general code to strip the ' - Character' because I know there will be other characters with the same problem. Putting in the brackets worked perfectly. Thanks.

[JimmXinu]

FYI, before people start reporting it to me as an FFDL problem, fanfiction.net appears to be down just at the moment.

[Jade Aislin]

Originally Posted by JimmXinu

FYI, before people start reporting it to me as an FFDL problem, fanfiction.net appears to be down just at the moment.

They had the same problem a couple days ago. I kept getting timed out. It lasted for little over an hour around 10pm my time, but eventually came back. I wonder if there doing some sort of revisions.

[Jade Aislin]

I've recently run into the problem of a story being on two different sites (http://www.fanfiction.net/s/8843117/1/ , http://archiveofourown.org/works/612...apters/1104270). It has the same title and same author, although one site has an extra chapter. When I tried to get the story from AO3 it tried to overwrite the FFN version which had more chapters. I don't usually notice when a story is updated. I only noticed because I knew that I had not gotten this story from AO3. If I had continued to get the story from AO3, I would have lost a chapter.

I realize this not a problem with the plugin itself, but I am unsure how I should handle it. Has anyone else had this problem? How did you deal with stories on mulitple sites?

Instead of immediately overwriting, would it be possible to compare the two epubs (for example look at word number) so that you could choose which one to keep? Or is there a way to create a new epub for each site so that I could compare the two manually?

[Jade Aislin]

Originally Posted by JimmXinu

Attached is a version that doesn't strip spaces from around the replace_metadata regular expressions. However, I can't stop the .ini parser stripping trailing spaces. So I've added a '\s'->' ' feature for the replacement string.

## So this works now:

replace_metadata:
 ships=> & =>/

That worked great.

Originally Posted by JimmXinu

## But to do the reverse, replace name/name with name & name,
## you have to do this:

replace_metadata:
 ships=>/=>\s&\s

Although, I can't think of a reason I would need to insert something with trailing spaces, I will keep that in mind.

And, yes, it took me this long to realize you had made the changes. I was too busy using your work around of brackets. Thanks.

[JimmXinu]

Originally Posted by Jade Aislin

I've recently run into the problem of a story being on two different sites (http://www.fanfiction.net/s/8843117/1/ , http://archiveofourown.org/works/612...apters/1104270). It has the same title and same author, although one site has an extra chapter. When I tried to get the story from AO3 it tried to overwrite the FFN version which had more chapters. I don't usually notice when a story is updated. I only noticed because I knew that I had not gotten this story from AO3. If I had continued to get the story from AO3, I would have lost a chapter.

I don't know how you got to that situation.

If you do an 'epub update' or 'update always', FFDL will refuse to update an existing epub that has more chapters than the site does.

So either you're trying to do an overwrite--which *can* overwrite an existing with more chapters (sometimes authors consolidate chapters), or you saw the Yes/No/ViewLog box and didn't realize that it said it *wouldn't* update it.

I suppose I could add an optional feature to only update books with matching URL, not title/author. But that could easily lead to duplicate books.

Or a different (optional?) feature, acting a bit like the reject list feature, to ask the user to confirm when an update/overwrite would change the story URL.

Do either of those ideas appeal to anyone?

[Jade Aislin]

Originally Posted by JimmXinu

I don't know how you got to that situation.

If you do an 'epub update' or 'update always', FFDL will refuse to update an existing epub that has more chapters than the site does.

So either you're trying to do an overwrite--which *can* overwrite an existing with more chapters (sometimes authors consolidate chapters), or you saw the Yes/No/ViewLog box and didn't realize that it said it *wouldn't* update it.

I suppose I could add an optional feature to only update books with matching URL, not title/author. But that could easily lead to duplicate books.

Or a different (optional?) feature, acting a bit like the reject list feature, to ask the user to confirm when an update/overwrite would change the story URL.

Do either of those ideas appeal to anyone?

I originally grabbed the story from FFN. I then tried to get the story from AO3. There had been at least three or four stories that came back 'updated', however, only two of them were rejected. So that url I provided may have not been overwriting the epub. Still, I want to look at the two versions of the same story before I delete one. Once I determine which one I will keep, I can simply add the other one to the rejection list.

I like the first option of updating based on story URL. Although it would create a duplicate, I could compare the duplicate stories and decide which one to keep. The second option of asking permission to overwrite the story URL could also work. It would let me know when I had a story on two sites. Then I could use the same method I am using now:

1. use different libraries to get the stories from different websites
2. merge libraries and allow for duplicates
3. examine duplicates to determine which one to keep
4. add rejected story URL to rejected list with 'duplicate story' tag

Either way, I'm still looking at duplicate stories to determine which one to keep. The first one would save me a step. It also might prevent duplicate stories when an author changes his or her name, as I have seen a few different authors on FFN change their name mid way through a story. The story would be recognized by the url and wouldn't be duplicated because the author's name no longer matched.

[JimmXinu]

Personally, if there's versions on two sites, I look at them on the sites and then choose rather than downloading both to decide.

Originally Posted by Jade Aislin

... It also might prevent duplicate stories when an author changes his or her name, as I have seen a few different authors on FFN change their name mid way through a story. The story would be recognized by the url and wouldn't be duplicated because the author's name no longer matched.

If the URL matches, FFDL will update that book, even if the title or author have changed.

Attached is a version that gives you the chance to decline updates that would change the URL of a story. Give it a try and let me know what you think--everybody, not just Jade Aislin.

UPDATE May 7, 2013 - Remove obsolete beta versions