[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by firefoxxy

I just tried to download this story: Spirit of Christmas
And I got the following comment from FFF: invalid group reference

I'm using the latest plugin. The story is online and I don't have it in my library yet. Do you need any further details?

It downloads fine for me, so yes, a debug log would help considerably.

[firefoxxy]

Hello Jim!

Since the error occurred with another story on AO3, I added it in the test run.
The story link is here: the second link

I attached the log file. Hope this helps.

[JimmXinu]

Originally Posted by firefoxxy

Since the error occurred with another story on AO3, I added it in the test run.
The story link is here: the second link

I attached the log file. Hope this helps.

This is showing that there's an error in one of your replace_metadata lines. "invalid group reference" means it's probably a \1, \2 etc in the replacement string without a watching () group in the regexp.

If you can't find it, I can look at your replace_metadata code. I will also investigate adding code to give a more specific error for this problem.

[firefoxxy]

Thank you Jim!

Yes, that solved it.

I had changed some lines in the replace_metadata just before downloading the stories.

I have this line in my personal.ini that you suggested:

ships_CHARS=>^(Uchiha|Uzumaki|Haruno|Hyuuga|Hatake|Namikaze) *([^ ]+)$=>\2

and wanted to try something similar on my own:

 genre=>^([aA]ngst) and (.*)$=>Angst\,\2
 genre=>^[mM]ild (.*)=>\2

This should have changed the tags to "Angst, *whatever*" and for the second tags it should strip "mild" from whatever was following - at least I thought that would happen. And since it was the second term that would change I thought it would be "2". Obviously it needs "1".
(My understanding of coding is clearly lacking.)

Thank you for your help, Jim!

[Amalthia]

Hi, I tried downloading a story from the hexfiles and the metadata is created, the chapters are there, but none of the text from the story is generated.

This is the link I'm using to download the story into calibre:
http://www.thehexfiles.net/viewstory.php?sid=125

I've attached the story to this comment so you can see what I'm seeing.

I'd appreciate any and all help in figuring this out.

Best Regards,
Amalthia

[JimmXinu]

Originally Posted by Amalthia

Hi, I tried downloading a story from the hexfiles and the metadata is created, the chapters are there, but none of the text from the story is generated.

This is the link I'm using to download the story into calibre:
http://www.thehexfiles.net/viewstory.php?sid=125

That site has rather poor HTML in the chapter pages. The newer HTML5 parser was having problems with it.

New test version posted (direct link)

2016-02-20
- Fix for adapter_thehexfilesnet missing chapter text.

[edeniz]

Just a quick but probably dumb question I have to ask for my peace of mind:

Regarding ship2chars.

I've had in my ini for a while now lines like this:

Spoiler:

  ships=>^John Sheppard\/Rodney McKay$=>Rodney McKay/John Sheppard

or:

 ships=>^Cordelia\/Xander$=>Xander Harris/Cordelia Chase

I've preferred to use ^ and $ instead of, for example, "\b" to make sure I don't get duplications or otherwise faulty renamings. Since I never noticed any problems with that, I assumed it would be ok. Now however, I have noticed a problem. It's thanks to this link:

http://archiveofourown.org/works/86535

I've downloaded it and later realized I already had it in my library. But, in doing so I also realized that in this new downloaded version I did not have anything in my characters field. Had it not been for the previously downloaded version, I would not have noticed this lack. Because, just the day before I'd downloaded a few fics from ffnet and was disappointed to realize that if the fic doesn't already have anything in the genre/character/ships fields, replace_metadata won't be able to add anything to them. (Yes, should be obvious, true, but apparently not. )

Strangely, this here:

Spoiler:

 ships=>^(.*)\/Rodney McKay$=>Rodney McKay/\1

is actually one of those things that work half the time and the other half not. No idea why.

Similarly, just this alone won't work either:

Spoiler:

 ships=>(.*)\/^Rodney McKay$=>Rodney McKay/\1

Anyway, tl;dr:
Does using ^$ in ships make the splitting of ships2chars impossible? (Has it always been so and I've just not noticed?) Or is it just me? Just so I know if I need to go over my ini and replace those with another regex.

[JimmXinu]

Originally Posted by edeniz

...
Anyway, tl;dr:
Does using ^$ in ships make the splitting of ships2chars impossible? (Has it always been so and I've just not noticed?) Or is it just me? Just so I know if I need to go over my ini and replace those with another regex.

This works perfectly well for me with that story:

[archiveofourown.org]

add_to_extra_valid_entries:,ships2chars
include_in_ships2chars:ships.NOREPL
# NOREPL prevents any ships patterns from being applied to ships2chars.

add_to_replace_metadata:
# ships=>(.*)\/^Rodney McKay$=>Rodney McKay/\1
# is never going to work because the '^' is misplaced
# I suggest this to get versions w/ & w/o spaces:
 ships=>^([^/&]+)[ ]*/[ ]*Rodney McKay$=>Rodney McKay/\1
 ships2chars=>[ ]*/[ ]*=>/
 ships2chars=>[ ]*&[ ]*=>/
 ships2chars=>([^/]+)/([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3\,\4
 ships2chars=>([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3
 ships2chars=>([^/]+)/(.+)=>\1\,\2

include_in_characters:characters,ships2chars

When things don't work sometimes I would look first for spaces that your patterns don't include, then differences in spelling, capitalization, etc.

Nitpick: You don't need to escape '/'s in the regexp patterns. So you can use just '/' vs '\/'. OTOH, I it doesn't hurt anything.

[Amalthia]

Originally Posted by JimmXinu

That site has rather poor HTML in the chapter pages. The newer HTML5 parser was having problems with it.

New test version posted (direct link)

2016-02-20
- Fix for adapter_thehexfilesnet missing chapter text.

Hi, I tested the updated plugin on this story and it worked!

Thank you so much for your help.

Amalthia

[apm]

I tried downloading from StoriesOnline.net, but I'm getting either "'NoneType' object has no attribute 'get_text'" (http://storiesonline.net/s/76055) or "'NoneType' object has no attribute 'find'" (http://storiesonline.net/s/46707).

Thanks,
apm

[mehetabelo]

APM: I just checked that link and (with latest release) I'm getting no problems. I do not have the test versions though, but I'm not sure that would cause a problem, as I didn't see any changes for storiesonline.

Now, the reason I logged in. I noticed a couple of stories that are available on finestories.com and not storiesonline.net. As I understand they are exactly the same site, though one (finestories) leaves out most of the sex stories. Is it possible, with a minor change, to include finestories as a download option?

I considered digging into the code and trying to add it myself, but that would mean spending a lot of time looking through things I don't grasp well. It may be a simple tweak that someone can do without a second thought, that's why I ask here before trying to figure it out myself.

[JimmXinu]

Originally Posted by apm

I tried downloading from StoriesOnline.net, but I'm getting either "'NoneType' object has no attribute 'get_text'" (http://storiesonline.net/s/76055) or "'NoneType' object has no attribute 'find'" (http://storiesonline.net/s/46707).

Both of those downloaded fine for me. Make sure you have the most recent released or test version.

If you continue to have problems, make sure that you are using the default site skin. After that, please run calibre in debug and post a log file if it still fails.

Originally Posted by mehetabelo

Now, the reason I logged in. I noticed a couple of stories that are available on finestories.com and not storiesonline.net.
...

FFF already contains code for finestories.com, but it apparently isn't working. And it is not terribly similar to the current code for storiesonline.net.

An initial look does suggest that finestories.com is similar to storiesonline.net now, but I'll have to spend some time looking more closely before I know for sure.

[apm]

Originally Posted by JimmXinu

Both of those downloaded fine for me. Make sure you have the most recent released or test version.

If you continue to have problems, make sure that you are using the default site skin. After that, please run calibre in debug and post a log file if it still fails.

I figured it out from the debug; there was a problem in my personal configuration file. Fixed it and it's working fine now. Thanks!

[edeniz]

Originally Posted by JimmXinu

When things don't work sometimes I would look first for spaces that your patterns don't include, then differences in spelling, capitalization, etc.

Maybe I overlooked something.

Originally Posted by JimmXinu

Nitpick: You don't need to escape '/'s in the regexp patterns. So you can use just '/' vs '\/'. OTOH, I it doesn't hurt anything.

Thanks again. But I think I initially escaped it because not doing so caused me problems. Don't remember what sort. Will have to try again and see.

Originally Posted by JimmXinu

include_in_ships2chars:ships.NOREPL
# NOREPL prevents any ships patterns from being applied to ships2chars.

Ok, that's how I understood it, especially since I've had to add characters lines as well if I wanted names in ships to be changed. But if that is right, basically lines like this:

Spoiler:

 ships=>^John Sheppard\/Rodney McKay$=>Rodney McKay/John Sheppard
 ships=>^Cordelia\/Xander$=>Xander Harris/Cordelia Chase

should pose no problem, shouldn't it? Because, again using this story as an example case, using a line like the above means it will make the replacement, but the ships will not be split into characters. My characters field remains empty.

[davidfor]

Originally Posted by mehetabelo

Now, the reason I logged in. I noticed a couple of stories that are available on finestories.com and not storiesonline.net. As I understand they are exactly the same site, though one (finestories) leaves out most of the sex stories. Is it possible, with a minor change, to include finestories as a download option?

The two sites have always shared a lot of the backend and I think the story database is the same, but I'm pretty sure that story submission is either separate, or has an option which to choose which site to put them on. It's been a whiles since I've looked, but I have found stories that were only on one, but would meet the rules of the other.

Originally Posted by JimmXinu

FFF already contains code for finestories.com, but it apparently isn't working. And it is not terribly similar to the current code for storiesonline.net.

An initial look does suggest that finestories.com is similar to storiesonline.net now, but I'll have to spend some time looking more closely before I know for sure.

The last time I looked at them, they were very similar, but just enough different that I didn't think it was worthwhile merging or otherwise reusing the code. I'll have a look and see what is going on now.