[GUI Plugin] FanFicFare

[JimmXinu]

I suspected that once you mentioned long text columns. However, I didn't see your issue when I tried it.

I'd approach it by disabling display of the long text column in Calibre Preferences -> Look & Feel -> Book Details

[Arkeus]

webnovel.com is back to not working.

[madarapotato]

webnovel.com not working please try to fix it ... thanks

[BookJunkieLI]

Originally Posted by JimmXinu

I suspected that once you mentioned long text columns. However, I didn't see your issue when I tried it.

I'd approach it by disabling display of the long text column in Calibre Preferences -> Look & Feel -> Book Details

Hey Jimm, I meant to say thank you last night for this tip. I hadn't known I could adjust what columns I wanted displayed in the book details so this is awesome to know.

[JimmXinu]

New Test Version Posted

2018-06-05
- Fixes for adapter_webnovelcom for site changes.

I've corrected the two faults I found. Kind in mind you should always include (SFW) story URLs when reporting issues if you want to make sure I see the same problems you do.

[madarapotato]

Originally Posted by JimmXinu

New Test Version Posted

2018-06-05
- Fixes for adapter_webnovelcom for site changes.

I've corrected the two faults I found. Kind in mind you should always include (SFW) story URLs when reporting issues if you want to make sure I see the same problems you do.

thank you it works now!!

[cryosaur]

Hello. I have a question about metadata normalization (replace_metadata, etc).

Is there a valid syntax for adding data to a metakey with a conditional check on another key? Sort of moving/transforming metadata between keys?

Some of my searching here and my personal knowledge of programming implies that this is possible. However, I can't seem to make it work. Yes, it does work if I actually search for something particular that I know is there for a particular fic; but it mysteriously fails if the pattern matches anything at all, which is needed to blindly add data. I suppose I can manually add to each adapter file a magic number to search on, which is then removed via exclude_metadata_post. ...

Well, that seemed like it worked but failed the full test. I added a magic value to the plugins's adapter file, and replace_ can use that to search on before it's cleaned up via exclude_post. However, I'm not sure but it seems that replace_metadata cannot find temporary (data not already on disk, read from website, but then exclude_post before write to disk) conditionalkey values, which... is rather odd. Is this working as intended?

[JimmXinu]

@cryosaur -- Can you give a concrete example of exactly what you want?

IE, story URL, and what text you want added to which metadata entry based on what value of what other metadata entry?

[JimmXinu]

New Test Version Posted

2018-06-06
- Add status to webnovelcom (#306) - Thanks, rikkitp

[skb]

Thank you for this plugin. It's wonderful.

A thought just occurred to me (sorry). Is it possible to show (somehow) the number of chapters in a downloaded fic?

[JimmXinu]

Originally Posted by skb

Thank you for this plugin. It's wonderful.

A thought just occurred to me (sorry). Is it possible to show (somehow) the number of chapters in a downloaded fic?

It depends on where you mean. I have a (numeric) Custom Column created in Calibre that I fill with the number of chapters. See FanFicFare Config, Custom Columns tab after creating the column in Calibre and restarting.

[catlover1019]

Hello, I'm having a problem downloading stories from inkbunny,net.

I'm using the Calibre plugin, and every story throws a list "index out of range" error.

Also, I don't know if this is related to the problem or not, but inkbunny.net now uses https://inkbunny.net/s/1234567 as the default URL format instead of https://inkbunny.net/submissionview.php?id=1234567 The older format still works and loads the correct page in a browser, but it has at least broken the finding URLs on a page feature. What I had to do was find the 171 URLs I wanted manually, put them in notepad and use find and replace to change the URL format. FanFicFare then recognized the URLs as valid, but doesn't seem to be able to actually get the files off of them. Like I said, both URL formats still work in a browser, but there might be a deeper change messing up FanFicFare.

[skb]

Originally Posted by JimmXinu

It depends on where you mean. I have a (numeric) Custom Column created in Calibre that I fill with the number of chapters. See FanFicFare Config, Custom Columns tab after creating the column in Calibre and restarting.

That's what I meant! Jim - you are a gem! Thank you.

[cryosaur]

Originally Posted by JimmXinu

Can you give a concrete example of exactly what you want?

Alright. I was not going to even bother submitting the earlier post as it was sort of a complaint that my idea of metadata normalization wasn't supported, but then I ran into what appears to be a bug.

Spoiler:

While the Warnings collection can be important to some sites (for searching), once in Calibre it has less value (IMO) as a separate field. For example, AO3's warnings of "Graphic Depictions Of Violence" and "Major Character Death" I change to tags of "Violence" and "Death". However, I still prefer to shunt the non-normalized Warnings data into a separate field so I can more-easily notice data not currently covered by my normalization scheme.

To normalize in this fashion, I have to "move" the metadata between the conditional key (warnings, etc.) and the target key (extratags, category, etc.), which might look something like this:

add_to_replace_metadata:
 extratags=>^(.*)$=>\1\,Violence&&warnings=>Graphic Depictions Of Violence

That code does not work though, because replace_matadata patterns will not find a match when looking for anything at all (^(.*)$). Not really sure why, but no matter. This limitation can be overcome by manually adding a "magic number"-type value to the target metakey. I'm not certain it would work in personal.ini as searching on ^(FanFiction)$ also failed, so I manually added one to the AO3's adapter file as a proof-of-concept test:

FanFicFare.zip\fanficfare\adapters\adapter_archiveofourownorg.py
...
        #magic search pattern
        self.story.addToList('freeformtags',"!")

        a = metasoup.find('dd',{'class':"freeform tags"})
        if a != None:
            genres = a.findAll('a',{'class':"tag"})
            for genre in genres:
                self.story.addToList('freeformtags',genre.string)
...

This does get around the limitation of not being able to match something non-specific, like so:

personal.ini
...
[archiveofourown.org]
add_to_replace_metadata:
 freeformtags=>^!$=>!\,Violence&&warnings=>Graphic Depictions Of Violence

exclude_metadata_post:
 warnings=~^(Graphic Depictions Of Violence)$
 freeformtags=~^!$

This sort of works. On an existing Calibre fic from AO3, the previously un-normalized Warning does successfully get transformed into a Tag during an Update Calibre Metadata from Website operation. However, if you do a full test with a new fic (or deleting and re-adding an existing story), this fails. How it fails is weird though. The "Graphic Depictions Of Violence" warning is not added to the Warnings field (is successfully excluded), but a "Violence" tag is not added to the Tags field (include_subject_tags: freeformtags).

I have not fully scrutinized the code behind replace_metadata, but this behavior leads me to suspect that the conditionalkey search might not be looking at all available data--both the current metadata in Calibre (if any), and the transitory metadata from the website (which may be excluded later). If so, I can't imagine that is working as intended, so it's a bug.

[JimmXinu]

@cryosaur -

First off, I'll tell you the same thing I tell everyone trying to normalize AO3 metadata: it's going to be a lot of work and it's never going to be perfect as long as AO3 allows authors to enter whatever they want.

As for the possible bug:

The way it works is that include/exclude_pre then replace_metadata then include/exclude_post are applied to each metadata entry as it's used, not all metadata as a whole.

So when you use warnings in a conditional clause, the pre, replace_metadata and post lines for warnings are applied to it before it is used.

I can see where that fact is non-intuitive. I'm going to look at the wiki pages and add some documentation about that.

There are ways to accomplish what you want even with that limitation:

Create a new metadata entry containing a copy of warnings that isn't edited for conditionals:

# Partial solution
[archiveofourown.org]
add_to_extra_valid_entries:,rawwarnings
include_in_rawwarnings:warnings.NOREPL

add_to_replace_metadata:
 freeformtags=>(.*)=>\1\,Violence&&rawwarnings=>Graphic Depictions Of Violence

exclude_metadata_post:
 warnings==Graphic Depictions Of Violence

But as you noticed, if there's nothing in freeformtags, the replacement isn't made.

IMO, it's better the create a new metadata entry containing a copy of warnings that you apply your edits and filters to that you include in freeformtags:

# More complete solution
[archiveofourown.org]
add_to_extra_valid_entries:,abbrevwarnings
include_in_abbrevwarnings:warnings.NOREPL
include_in_freeformtags:freeformtags,abbrevwarnings

add_to_replace_metadata:
 abbrevwarnings=>^Graphic Depictions Of Violence$=>Violence
 abbrevwarnings=>^Major Character Death$=>Death

include_metadata_post:
 abbrevwarnings==Violence
 abbrevwarnings==Death
 
exclude_metadata_post:
 warnings==Graphic Depictions Of Violence
 warnings==Major Character Death