[GUI Plugin] FanFicFare

[saluli]

The new version worked very well. Thanks for it!

@DavidTC

I spent a lot of time trying to tame AO3 tags to my liking, but the tags are winning.... In some cases I managed to create a working Regex, but usually after hours reading tons of tutorials... Some of the more complex things end being faster to do by hand than to try to get a working regex....

If you manage to find a good way to do it, please share with us!

[DavidTC]

Originally Posted by JimmXinu

Actually, AO3 will let you make a story with both '/' and '&' in the same ship tag. With or without space around either. Yay for freeform tags, I guess?

Well, yeah, you can do it. I'm just saying I (and most people) would really have no idea what that means. If I actually ran across X&Y/Z, I think I would have to mentally parse it as X&(Y/Z), aka, X being friends with the couple Y/Z. Alternately, it means (X&Y)/Z, as in, Z is in a relationship with X, and in a relationship with Y, and X and Y are just friends?

There does not appear to be any standard defined order of operations there. This obviously would cause havoc with *sorting* the tags. (Actually, even with a defined order of operations, it would get complicated.)

However, as combining multiple separators in the same tag doesn't really seem to exist in the wild (Probably because no one would understand it.), I was just suggesting that FFF just give up if it sees two different types, instead of trying to figure it out. (Although it could still run the names past regexps to normalize them.)

[DavidTC]

Originally Posted by JimmXinu

What sites use '/' for romantic ships and '&' for platonic? I don't recall seeing that before. A few example story URLs would also help.

Well, I can just find AO3 ones, like http://archiveofourown.org/works/2824976

Some sites don't do friendship relationships, and some just do *all* relationships the same way.

But I'm pretty certain *if* they are distinguished, that's how they usually are distinguished. / has a long tradition going back to the origins of fanfic for relationships, and & seems like an obvious choice of grouping two characters that isn't that. (Although I wouldn't be surprised to see +.)

Originally Posted by JimmXinu

I want to explore the sites that use that a bit before making any decisions. What happens in cases where it's common to have a '/' as part of the name, such as superhero fandoms?

I'm not sure of this question. How is this handled now?

If the site uses / as a separator, and also puts / in names, then sort_ship simply can't work at all, from what I can tell. That's not changed with what I suggested. (I see no solution for that silliness at all.)

If they use / in names and *something else* like | as a separator, well, that sounds like a job for regular expressions changing things going in, and then changing them later. (My problem is I've go *two* different types and I'm trying to end up with two different types. If I just had &, I'd turn them into /, and then back to &.)

Or alternately you could let people list the separators in an option, which would let them list | for that site, and & and / for AO3, and so on.

[DavidTC]

Originally Posted by saluli

The new version worked very well. Thanks for it!
I spent a lot of time trying to tame AO3 tags to my liking, but the tags are winning.... In some cases I managed to create a working Regex, but usually after hours reading tons of tutorials... Some of the more complex things end being faster to do by hand than to try to get a working regex....

Oh, don't get me started on AO3 'tags', of which a third appear to be goofy stream-of-consciousness conversations with the reader. I'm just trying to normalize characters and relationships at this point.

*Tags* might, frankly, be me attempting to recognize ten or twenty standardish words like 'Angst' and then erasing everything else.

I have actually considered putting up some sort of wiki with pages saying 'Here are the config parts that should normalize all names in Fandom X'.

[JimmXinu]

Originally Posted by DavidTC

I have actually considered putting up some sort of wiki with pages saying 'Here are the config parts that should normalize all names in Fandom X'.

Please do. I'd be happy to put a page under the project wiki for it. (I think anyone with a GitHub account can edit wiki pages.) I've asked before for the people who spend lots of time normalizing tags to share their work with little result.


I've posted a new Test version (direct link) that fixes the problem you reported as well as adding sort_ships_splits and ships_CHARS features.

As requested, you can use replace_metadata on ships_CHARS to operate on the individual parts of ships entries while they are split.

You can use sort_ships_splits to specify more than one regular expression to use to split by and what string to use to join the char parts back together after sorting.

Here are my test settings for it:

Spoiler:

[defaults]
add_to_replace_metadata:
 ships_CHARS=>Thing=>Object
 characters,ships_CHARS=>Bob=>Wobert

sort_ships:true

## Each line indicates first a regex that should be used to split each
## ships entry and then, after => the string to use to merge the parts
## back together.  \s == blank space.
## Each part will have replace_metadata with key ships_CHARS applied.
## By default, sort_ships_splits:/=>/
sort_ships_splits:
 [ ]*/[ ]*=>\s/\s
 [ ]*&[ ]*=>\s&\s

## here's my tester:
## story URL: test1.com?sid=1002
[teststory:1002]
add_to_category_list:Supernatural
#add_to_category_list:abc,bcd,cde
add_to_genre_list:Action,Romance
chaptertitles:Oneshot,Twoshot
#Prologue,Chapter 1\, Xenos on Cinnabar,Chapter 2\, Sinmay on Kintikin,3. Chapter 3,Chapter 4,Chapter 5,Chapter 6,Chapter 7,Chapter 8
add_to_ships_list:Thor/Jane,Bob,Bob/Thor,Merlin/Arthur,Thing 1/Thing 2,A/B/C,Z/Y/X/V,qewrty / adsf,Harry & Amelia,Ron&Neville&Trevor,Z/A&Y/B
add_to_characters_list:Fred Weasley,George,Bob,Harry
rating:G
Status:Completed
#title:Some Other Title

I had to shift some code around to make this work correctly, so no promises it doesn't break something else. I don't think it does...

(And anybody who asks for sorting by last name instead of string order will be roundly ignored.)

[Tanjamuse]

I would love to share the very long list of replace metadata I have, I just don't know where and how, I could also with time add to it with other fandoms than the ones I need, but it's only Characters, Fandoms and ships.

[edeniz]

I'm trying to make a custom column in calibre in my fanfiction library. I want to separate pairings and genre (ao3's freeformtags nothwithstanding) to its own column, and I think I've managed it. Only, the column is now empty. I'd like to get the tags needed from the metadata already downloaded into the tags column instead of burdening the servers with a probably unnecessary metadata download, but I'm not sure how to do so.

Can anyone help me out a little here?

Regarding the above mentioned sharing: I'd be willing to contribute as well, but there is something I've noticed about the replace_metadata to consider: There is only a limited amount of metadata replacing possible. (What's that number anyway?) Case in point: I dislike fanfiction.net's shortening of the surnames and use replace_metadata to get rid of it. But I've had to prepare different personal.ini for each fandom I download from because when I tried to have it for two fandoms in the same personal.ini I got an error message that it was too long. Ao3's freeformtags are also annoying sometimes when people use it for commentary instead of tagging, but because I did not want to prepare yet another personal.ini for that, I had to opt to using calibre's tag mapper for it.

[JimmXinu]

I've added a new wiki page MetadataManagement and put a couple of my metadata settings in it. I think anyone with a GitHub account(free) can add to it.

Originally Posted by edeniz

I want to separate pairings and genre (ao3's freeformtags nothwithstanding) to its own column, and I think I've managed it. Only, the column is now empty. I'd like to get the tags needed from the metadata already downloaded into the tags column instead of burdening the servers with a probably unnecessary metadata download, but I'm not sure how to do so.

I can't help with your column being empty without specifics. If you've configured a 'Saved Metadata Column' (see bottom of 'Custom Columns' tab in FFF config), the plugin will have saved the metadata for each book you've downloaded/updated since saving that setting.

You can then use 'Update Calibre Metadata from Saved Metadata Column'. If any of the books you're updating don't have saved metadata, FFF will perform 'Update Calibre Metadata from Web Site' for them and save the metadata in the column.

Originally Posted by edeniz

...
There is only a limited amount of metadata replacing possible. (What's that number anyway?)
...

I've never seen such a limit. Can you post the error message it gives you and an example of a personal.ini that causes it?

[edeniz]

Originally Posted by JimmXinu

I've added a new wiki page MetadataManagement and put a couple of my metadata settings in it. I think anyone with a GitHub account(free) can add to it.



I can't help with your column being empty without specifics. If you've configured a 'Saved Metadata Column' (see bottom of 'Custom Columns' tab in FFF config), the plugin will have saved the metadata for each book you've downloaded/updated since saving that setting.

You can then use 'Update Calibre Metadata from Saved Metadata Column'. If any of the books you're updating don't have saved metadata, FFF will perform 'Update Calibre Metadata from Web Site' for them and save the metadata in the column.



I've never seen such a limit. Can you post the error message it gives you and an example of a personal.ini that causes it?

Well, I created the Pairings column to be "comma separated text, like tags, shown in the tag browser", with the lookup name pairings. Same with the Genre column, only with the lookup name genre. That's really the only info I can give about it. I didn't do anything special. But I guess that's moot because from what you've said I made a big mistake previously.


I had not configured a "saved metadata column". For some reason I never even saw that line in the fanficfare customization settings. Fixed it now.

Oh well. I'll have to do it manually then. After all, there are only a handful of stories with pairings in my as yet small library, and I want only the handful of main genres in my genre column anyway, so it shouldn't take too long.

As for the limit thing, I'm attaching the personal.ini like you said. And a screenshot of the error message, since it's too long and I can't find a log of it (if it exists).

[JimmXinu]

Originally Posted by edeniz

As for the limit thing, I'm attaching the personal.ini like you said. And a screenshot of the error message, since it's too long and I can't find a log of it (if it exists).

I don't know why you started with a copy of defaults.ini. That will occasionally cause problems when you don't pick up changes made to defaults.ini because you're already overriding them.

The problem is this bad line, not any arbitrary length limit:

 characters=>Ameyuri R\.=>=>Ringo Ameyuri&&category=>Naruto&&category=>Naruto

But the error message is vague, so I can see why you came to that conclusion. I'll try to improve it.

(FYI, you have several lines in your replace_metadata there that are character instead of characters.)

[DavidTC]

Originally Posted by JimmXinu

I've posted a new Test version (direct link) that fixes the problem you reported as well as adding sort_ships_splits and ships_CHARS features.

As requested, you can use replace_metadata on ships_CHARS to operate on the individual parts of ships entries while they are split.

You can use sort_ships_splits to specify more than one regular expression to use to split by and what string to use to join the char parts back together after sorting.

This all seems to be working. Thanks. I'm tracing down various ships of different lists and orders, making sure the regexps line up, and then updating, and it all seems to working correctly.

Originally Posted by JimmXinu

Please do. I'd be happy to put a page under the project wiki for it. (I think anyone with a GitHub account can edit wiki pages.) I've asked before for the people who spend lots of time normalizing tags to share their work with little result.

Great! I'll start trying to collect some of my more generic ones, and put them up. (Fun fact: AO3 tags for characters often include the word (Character) after them. Yeah, uh, thanks AO3, I was confused as to what tag that was in the *character* list.)

Then I can put some fandom-specific ones, I guess. That's going to be a bit more subjective, but I guess most people would be okay going to full names, and if not, that's easy enough for them to change without having to deal with the regexp, which is the most complicated part.

[edeniz]

Originally Posted by JimmXinu

I don't know why you started with a copy of defaults.ini. That will occasionally cause problems when you don't pick up changes made to defaults.ini because you're already overriding them.

I did that because I thought this would make it easier for me to compare it to defaults.ini whenever it updates. Thought I'd just highlight the differences and compare. If it doesn't work someday for me or becomes cumbersome I'll trim it down. For now it works.

The problem is this bad line, not any arbitrary length limit:

 characters=>Ameyuri R\.=>=>Ringo Ameyuri&&category=>Naruto&&category=>Naruto

But the error message is vague, so I can see why you came to that conclusion. I'll try to improve it.

(FYI, you have several lines in your replace_metadata there that are character instead of characters.)

Oh, I see. Thank you!

At first I went over it several times, to see if there was an error in there somewhere, but somehow never picked up on those lines. That and the error message telling me it was too long is why I thought there was a length limit. And so I cut off most of it, leaving only the most often used ones. Guess I removed also those bad lines when I did that, so I never noticed it wasn't a length limit that was the problem.


Forgot to say this before: Thank you for the link to the wiki page, I'll be certain add my tags to the list. I think it's awesome and very helpful that people are pooling their tag change regex together.

[Montana Harper]

Originally Posted by DavidTC

(Fun fact: AO3 tags for characters often include the word (Character) after them. Yeah, uh, thanks AO3, I was confused as to what tag that was in the *character* list.)

As I understand it, that happens when there are identical tags in multiple categories. For instance, you can have Harry Potter as a fandom name, but also Harry Potter as a character name. There has to be a way to differentiate the two, and AO3 has chosen to add an identifier like that. You'll also find "- Fandom" after some tags.

[eschwartz]

Originally Posted by edeniz

I did that because I thought this would make it easier for me to compare it to defaults.ini whenever it updates. Thought I'd just highlight the differences and compare. If it doesn't work someday for me or becomes cumbersome I'll trim it down. For now it works.

The best way to compare changes to the defaults is to take a look at the file's revision history: https://github.com/JimmXinu/FanFicFa...n-defaults.ini

(FanFicFare repo --> Select file --> click "History")

[edeniz]

Originally Posted by eschwartz

The best way to compare changes to the defaults is to take a look at the file's revision history: https://github.com/JimmXinu/FanFicFa...n-defaults.ini

(FanFicFare repo --> Select file --> click "History")

Thank you! I hadn't realized. I still keep finding out new things about fanficfare and calibre that I didn't know (and half the time those seem to be common sense *sigh*).