[GUI Plugin] FanFictionDownLoader

[JimmXinu]

It looks like fanfiktion.de has changed the URLs they use for their own stories a little bit, adding the story name after the chapter number.

Try the attached version, it should works better for you.

UPDATE Jul 23, 2013 - Remove obsolete beta versions

[Jade Aislin]

I have most of the coding done for my end tags, but I had a few questions.

1. In order to get the end tags to the end of a ship when there is more than one end tag, I had to duplicate a couple lines of code.

 ships=> (\(Family\)|\(Friendship\))(.*?)$=>\2 \1
 ships=> (\(Family\)|\(Friendship\))(.*?)$=>\2 \1 

 ships_LIST=>(\(Family\)|\(Friendship\))( )?(.*?),=>\3 \1,
 ships_LIST=>(\(Family\)|\(Friendship\))( )?(.*?),=>\3 \1,

Is there a way to tell it to redo a line of code until it no longer finds what the code is looking for?

2. When I put in code for putting the end tag at the end for the final ship in the list, I had to delete a space from the code to make it work.

 ships_LIST=>,(.*?)( \(Family\)| \(Friendship\))/(.*?)$=>,\1/\3\2
 ships_LIST=>,(.*?)(\(Family\)|\(Friendship\))/(.*?)$=>,\1/\3 \2

Originally I tried to put in ( )? to show that the space might not be there.

ships_LIST=>,(.*?)(( )?\(Family\)|( )?\(Friendship\))/(.*?)$=>,\1/\3 \2

However, I received an unmatched group error when I added the '( )?'.

Is there a way I can streamline these two lines of code?

3. Finally, I used this code to separate end tags into different ships:

 ships_LIST=>(.*?) \((.*?)\) \((.*?)\),=>\1 (\2), \1 (\3),
 ships_LIST=>,(.*?) \((.*?)\) \((.*?)\)$=>,\1 (\2), \1 (\3)

The first line worked fine and 'A/B (Friend) (Family)' became: 'A/B (Friend), A/B (Family)'.
The second line, however, deleted the second end tag. 'C/D (Friend) (Family)' became: 'C/D (Friend)'.

I'm not sure why it deleted the second end tag. I also tested '\1 (\3), \1 (\2)' and still had the second tag deleted. I'm not really sure how to fix the last line to get the last ship in the list fixed.

Any help would be appreciated.

[JimmXinu]

Originally Posted by Jade Aislin

I have most of the coding done for my end tags, but I had a few questions.

1. In order to get the end tags to the end of a ship when there is more than one end tag, I had to duplicate a couple lines of code.
...
Is there a way to tell it to redo a line of code until it no longer finds what the code is looking for?

No, there isn't. And I'm not inclined to add such a feature. I expect most users would end up creating infinite loops more often than not.

Since you're matching the rest of the entry too, I suspect multiple lines may be as good as it gets.

Originally Posted by Jade Aislin

2. When I put in code for putting the end tag at the end for the final ship in the list, I had to delete a space from the code to make it work.
...
Originally I tried to put in ( )? to show that the space might not be there.
...
However, I received an unmatched group error when I added the '( )?'.

Is there a way I can streamline these two lines of code?

Python regex has what is IMHO, a flaw. Unmatched group replacements (\1,\2,etc) instead of being ignore throw exceptions. There's a work around for it--'(| )' I think in this case--and it's been discussed in this thread in past. Google will also find various workarounds and comments.

As for streamlining, instead of two lines, one ending in ", " and one with "$", try "(, |$)".

Originally Posted by Jade Aislin

3. Finally, I used this code to separate end tags into different ships:
...
I'm not sure why it deleted the second end tag. I also tested '\1 (\3), \1 (\2)' and still had the second tag deleted. I'm not really sure how to fix the last line to get the last ship in the list fixed.

I suspect that combining them into one line with "(, |$)" will help.

[Firedancer885]

Originally Posted by JimmXinu

It looks like fanfiktion.de has changed the URLs they use for their own stories a little bit, adding the story name after the chapter number.

Try the attached version, it should works better for you.

Yesss

[Jade Aislin]

Originally Posted by JimmXinu

There's a work around for it--'(| )' I think in this case--and it's been discussed in this thread in past. Google will also find various workarounds and comments.

I still seemed to get the unmatched group, but as I was able to get it to work without that, I'll look into this problem later.

Originally Posted by JimmXinu

As for streamlining, instead of two lines, one ending in ", " and one with "$", try "(, |$)".

I suspect that combining them into one line with "(, |$)" will help.

That did help. However, I decided not list all the end tags and use (.*?) in my code. Then I added the next end tag and got a strange response in the final ship of the list.

The first line of code works for all other ships. It looks for a word in parenthesis and then places it at the end of the ship.

 ships_LIST=>\((.*?)\)( )?(.*?),=>\3 (\1),
 ships_LIST=>\((.*?)\)( )?(.*?)$=>\3 (\1)

However the second line, used for the last ship in the list, is moving the parenthesis tag, but it is not getting rid of the original parenthesis. My output goes from 'O (One-Sided) (Family)/P' to 'O (One-Sided) (Family)/P (One-Sided)'.

Is there a reason that the last line of code is moving the tag I want, but is not deleting it from the original position?

[JimmXinu]

Originally Posted by Jade Aislin

 ships_LIST=>\((.*?)\)( )?(.*?),=>\3 (\1),
 ships_LIST=>\((.*?)\)( )?(.*?)$=>\3 (\1)

However the second line, used for the last ship in the list, is moving the parenthesis tag, but it is not getting rid of the original parenthesis. My output goes from 'O (One-Sided) (Family)/P' to 'O (One-Sided) (Family)/P (One-Sided)'.

Is there a reason that the last line of code is moving the tag I want, but is not deleting it from the original position?

\((.*?)\) matches (non-greedy) (something)

(.*?) matches (non-greedy) anything--including additional (something) strings.

I'd experiment with using something like ([^\(]*?)--match (non-greedy) anything except open paren.

Frankly, there's a lot of trial and error in using complex regular expressions, even for experienced users.

[JOJO1]

Originally Posted by JimmXinu

It looks like fanfiktion.de has changed the URLs they use for their own stories a little bit, adding the story name after the chapter number.

Try the attached version, it should works better for you.

Hi Jimm,

thank you for help.





Perfect!!

[JimmXinu]

Version 1.7.32 - 23 Jul 2013

• Fixes for fanfiktion.de & thehexfiles.net, add entries to teststory valid list.

[JimmXinu]

Latest version has a bug with saving plugin settings on calibre prior to 0.9.39. This version corrects it.

UPDATE Jul 28, 2013 - Remove obsolete beta versions

[JOJO1]

Originally Posted by JimmXinu

Latest version has a bug with saving plugin settings on calibre prior to 0.9.39. This version corrects it.

Hallo Jimm,

now there is another problem with "fanfiktion.de". The downloader add the right Story, bud the Metadatas are every time from the first Story of the author.

I think i need your help again.

[JimmXinu]

Originally Posted by JOJO1

now there is another problem with "fanfiktion.de". The downloader add the right Story, bud the Metadatas are every time from the first Story of the author.

Actually, it's metadata other than title/author (I almost stopped looking when I saw the title/author was correct on each), and it's taking the summary, rating, etc from the last story of the author, not the first.

This is another problem caused by fanfiktion.de changing their URLs to include story name. The attached version should fix it.

UPDATE Jul 28, 2013 - Remove obsolete beta versions

[JOJO1]

Originally Posted by JimmXinu

Actually, it's metadata other than title/author (I almost stopped looking when I saw the title/author was correct on each), and it's taking the summary, rating, etc from the last story of the author, not the first.

This is another problem caused by fanfiktion.de changing their URLs to include story name. The attached version should fix it.

It works perfekt.

Greetings from Germany

Thank You

[SallyK]

I'm having problems with a couple of stories from Stories of Arda (http://www.storiesofarda.com), both by the same author.

http://www.storiesofarda.com/chapter...w.asp?SID=4018 comes up with the error message - 'NoneType' object has no attribute 'find'
whereas http://www.storiesofarda.com/chapter...w.asp?SID=5046 seems to add okay, and you get the book structure with the chapter headings, but no actual text of the story.

I tried http://www.storiesofarda.com/chapter...w.asp?SID=6731 by another author, and it worked fine.

Is it a problem with the website, or is there anything I can do to fix it? Thank you for any help you can give.

[JimmXinu]

Originally Posted by SallyK

I'm having problems with a couple of stories from Stories of Arda (http://www.storiesofarda.com), both by the same author.

http://www.storiesofarda.com/chapter...w.asp?SID=4018 comes up with the error message - 'NoneType' object has no attribute 'find'

Interesting. Chapter 8--and only chapter 8--of that story throws out an "are you an adult?" question. We didn't know that site ever required that. I can make the adapter work with that when is_adult is set, but because it's on chapters, it can't stop and prompt the user.

However, for that particular story, that only gets us as far as the next issue:

Originally Posted by SallyK

whereas http://www.storiesofarda.com/chapter...w.asp?SID=5046 seems to add okay, and you get the book structure with the chapter headings, but no actual text of the story.

Both this and the story above have extra <body> tags inside the story HTML that's confusing the parser. That's a problem with the HTML as uploaded by the author. But it's one I've seen before on other sites, and I've applied the same fix.

However, I need you to test several stories by other authors on storiesofarda.com to make sure it didn't break any of them. (I don't read storiesofarda.com.)

Attached is a test version with fixes.

UPDATE Jul 28, 2013 - Remove obsolete beta versions

[SallyK]

Thank you very much for getting on it so promptly. I've tried your version with the stories I already listed, and several others from the archive by different authors, and they all seem to be fine. (I haven't read every word, but I've scrolled through them and not seen any errors.)