[GUI Plugin] FanFicFare

[ownedbycats]

Originally Posted by ackomb

The character is Addison Montgomery which I change into Addison Forbes Montgomery

Usually I use below code, but it doesn't work since Grey's Anatomy isn't tagged as a show

 characters=>^Addison Montgomery$=>Addison Forbes Montgomery&&category==Grey's Anatomy Universe

Just curious, is that name used by a different character in any other of your fandoms? if not, I'd just strip the conditional.

[PacifiedOwl]

Originally Posted by JimmXinu

Question for Plugin Users

A number of FFF's settings and defaults come from the early days ~10 years ago when users didn't trust FFF to set Calibre metadata and wanted options not to.

There are several options available on every download/update:

1. Output Format
2. Update Mode
3. Update Calibre Metadata
4. Update EPUB Cover
5. Background Metadata (updates only)

Most are self explanatory. 'Update Calibre Metadata' if unchecked, prevents FFF from updating any Calibre metadata--except for the explicit 'Update Calibre Metadata' modes which do anyway.

Which are these settings are actually useful today on every download/update?

Personally, the only one I ever change is Update Mode.

For example, does anyone run with 'Update Calibre Metadata' unchecked? Or change output format on individual downloads?

I use Update Mode very frequently, I don't think I've ever used the others.

[JimmXinu]

Originally Posted by ackomb

I have another mystery (to me at least) I have "Keep Existing Tags when Updating Metadata" checked and all "New Only" boxes unchecked. So there is this story where I've manually amended the character cause it's a character from an untagged show which is why my regex setup isn't working, no biggy. However every time I update the story it adding the old tag and removes the tag I've added.

So I'm guessing there is some setting I have wrong, but I can't figure out which.

I don't want to check "New Only" in the customs columns because that would mean I don't get the tags that might be added by the writer after the initial download.

An example for a story is:https://archiveofourown.org/works/28121997

The character is Addison Montgomery which I change into Addison Forbes Montgomery

Usually I use below code, but it doesn't work since Grey's Anatomy isn't tagged as a show

 characters=>^Addison Montgomery$=>Addison Forbes Montgomery&&category==Grey's Anatomy Universe

What I want it do do is keep the tags I've added or amended but update any new ones added by the writer.

You seem to be implying ("I don't want to check "New Only" in the customs columns...") that you're actually setting to a custom column.

"Keep Existing Tags when Updating Metadata" only applies to Calibre's Tags column--not custom columns of the tags type.

Re: Your replace_metadata--that story doesn't have "Grey's Anatomy Universe" in fandom (aka category for AO3), therefore it doesn't match the conditional.

Originally Posted by Zerubel

Hello all! I've been lurking and reading here for a while and wanted to ask if there is, perhaps, a more comprehensive page or wiki somewhere detailing normalizing tags between ffnet and ao3
...

Good luck with that. I stand by my opinion from 10-25-2020:

Originally Posted by JimmXinu

...
AO3 tag normalizing/wrangling comes up over and over again. I'm not going to go over all of it again, but AO3's decision to allow authors to input basically anything--and any number of anything--as tags makes normalizing problematic at best.

Personally, I consider it futile. But I will help with specific problems.

[Zerubel]

Originally Posted by JimmXinu

Good luck with that. I stand by my opinion from 10-25-2020:

Oh boy XD well i think i've made a decent (if perhaps messy and long) start to normalizing tags, even made a spreadsheet to join text so i just have to plug in the tags, but i couldn't get two sets to work:

category=>Fullmetal Alchemist - All Media Types=>Fullmetal Alchemist
category=>Fullmetal Alchemist (Anime 2003)=>Fullmetal Alchemist
category=>Fullmetal Alchemist: Brotherhood & Manga=>Fullmetal Alchemist

category=>Pokémon=>Pokemon
category=>Pocket Monsters | Pokemon (Anime)=>Pokemon
category=>Pocket Monsters | Pokemon (Main Video Game Series)=>Pokemon
category=>Pocket Monsters: FireRed & LeafGreen | Pokemon FireRed & LeafGreen Versions=>Pokemon
category=>Pocket Monsters: Red & Green & Blue & Yellow | Pokemon Red Green Blue Yellow Versions=>Pokemon
category=>Pocket Monsters: Sun & Moon | Pokemon Sun & Moon Versions=>Pokemon

the first 'Pokemon' one worked (in fact most if not all of my ffnet normalizations have worked) but the rest remain stubbornly the same. I'm assuming there's something about spaces that i'm missing - but aside from copying folks' answers (Harry P\. to Harry Potter, for instance) i'm not sure what's wrong?

(and as an aside, the image is the spreadsheet - am i doing this the messy/bloated way, or is this correct?)

[JimmXinu]

Originally Posted by Zerubel

Oh boy XD well i think i've made a decent (if perhaps messy and long) start to normalizing tags, even made a spreadsheet to join text so i just have to plug in the tags, but i couldn't get two sets to work:

First, you need to escape characters that regexp uses including ()|. And internally, FFF carries & as & so use that in place of just &. Search for 'regular expression escaping' or tutorials to learn about regexp.

If that doesn't fix it, you will also need to give me story URL(s) that demonstrate the problem(s).

[ackomb]

Originally Posted by ownedbycats

Just curious, is that name used by a different character in any other of your fandoms? if not, I'd just strip the conditional.

Not this one specifically but others yes, try Alex it's in at lease 3 fandoms I follow.

[leckadams]

Not sure if anyone else is getting this error, but it started happening again to me yesterday late evening & I didn't think anything of it, but it's happening today as well.


Status
Title
Author
Comment
URL
Error
An Unexpected Companion
ANGSWIN
cloudscraper reports: "Detected a Cloudflare version 2 challenge, This feature is not available...."
https://www.fanfiction.net/s/1387577...cted-Companion

[ackomb]

Originally Posted by JimmXinu

"Keep Existing Tags when Updating Metadata" only applies to Calibre's Tags column--not custom columns of the tags type.

I thought that was the case, but I wasn't sure.

Originally Posted by JimmXinu

You seem to be implying ("I don't want to check "New Only" in the customs columns...") that you're actually setting to a custom column.

The reason I don't want to check "New Only" is because I still want it to add newly added tags, I just don't want it to remove the ones I've added when I update a story. Since I'm assuming "New Only" applies to a new story and not a new tag.

But if it isn't possible that's okay, it just means I can stop looking and find another solution.

[Zerubel]

Originally Posted by JimmXinu

First, you need to escape characters that regexp uses including ()|. And internally, FFF carries & as & so use that in place of just &. Search for 'regular expression escaping' or tutorials to learn about regexp.

If that doesn't fix it, you will also need to give me story URL(s) that demonstrate the problem(s).

Oh, thank you! I'll put some time into that after I'm off work today.

My second question - is it possible to tell Fanficfare to put tags like characters and pairings into custom columns and not add them to the epubs main tags? I've managed to do it with ao3's "genre" (aka freeform madness) into a text-only column, but if possible i'd still like to collect that data to browse characters and pairings without adding to the main clutter.

I experimented a bit with a non-fanfiction book - i have a custom column for Genre in which i put "Fantasy, Isekai" and removed them from the main tags - the tag browser on the side shows them in genre and they do not appear on the main tags like 'Light Novel (Ranobe)' still does.

[Zu'l]

I'm having same problem when trying to download fic from Royal Road (link https://www.royalroad.com/fiction/24709), how do you tell which chapter(s) are causing the 'maximum recursion depth exceeded' issue?

[JimmXinu]

Originally Posted by Zerubel

My second question - is it possible to tell Fanficfare to put tags like characters and pairings into custom columns and not add them to the epubs main tags? I've managed to do it with ao3's "genre" (aka freeform madness) into a text-only column, but if possible i'd still like to collect that data to browse characters and pairings without adding to the main clutter.

Yes. There's the GUI Custom Columns tab and the personal.ini custom_columns_settings setting.

For historical reasons, the personal.ini setting that controls what goes into Calibre's Tags column is include_subject_tags. Some sites also use extra_subject_tags to add to it (from before we had add_to_*).

Originally Posted by Zu'l

I'm having same problem when trying to download fic from Royal Road (link https://www.royalroad.com/fiction/24709), how do you tell which chapter(s) are causing the 'maximum recursion depth exceeded' issue?

Run in debug mode and look at the log, or set continue_on_chapter_error:true and look to see which chapter is marked ERROR in the ebook..

Originally Posted by ackomb

The reason I don't want to check "New Only" is because I still want it to add newly added tags, I just don't want it to remove the ones I've added when I update a story. Since I'm assuming "New Only" applies to a new story and not a new tag.

You can use a custom_columns_settings line with the a flag set. From defaults.ini:

## (Plugin Only) - You can also populate calibre custom columns with
## the site specific metadata using custom_columns_settings (but only
## if 'Allow custom_columns_settings from personal.ini' is checked in
## the plugin GUI config.)  There are three parts, the entry name,
## then the label of the calibre custom column, then (optionally) a
## 'mode'.  'r' to Replace any existing values, 'a' to Add to existing
## value (use with tag-like columns), and 'n' for setting on New books
## only.  (Default is 'r'.)

## Literal strings can be set into custom columns using double quotes.
## Each metadata=>column mapping must be on a separate line and each
## needs to have one space at the start of each line.

## 'r_anthaver' and 'n_anthaver' can be used to indicate the same as
## 'r' and 'n' for normal downloads, but to average the metadata for
## the differents story in an anthology before setting in integer and
## float type custom columns.  This can be useful for a averrating
## column, for example.  Default is to sum the values of all stories,
## and numChapters and numWords are always summed.

#custom_columns_settings:
# cliches=>#acolumn
# themes=>#bcolumn,a
# timeline=>#ccolumn,n
# "FanFiction"=>#collection
# averrating=>#averrating,r_anthaver

[Science Friction]

I'm also getting both IAUA and v2 challenge errors from ff.net.

I notice that archive.org has a crawl of ff.net dating to 2017 or so, and there are some newer ones around. I wonder if fanficfare could retrieve from one of those instead. They are stored on archive.org in large zip files but maybe it is feasible to retrieve the appropriate byte ranges with the python zip library.

[ownedbycats]

FictionPress was working on an app update a few hours ago, so I'd try again a little later and see if it clears up.

[JimmXinu]

New Test Version Posted

2021-05-06
- Fix XF authorUrl and author_avatar_cover feature. Closes #695

[Science Friction]

Originally Posted by ownedbycats

FictionPress was working on an app update a few hours ago, so I'd try again a little later and see if it clears up.

Got v2 challenge just a minute ago, will try again later, but I think we should just assume that ffnet wants to lock people into their software, so the download situation will get worse rather than better. I do see that fanficfare was able to retrieve the metadata page of the story I tried to download, but got errors for the individual chapter pages. I also see that the reddit fanfic bot is still able to post (and therefore presumably retrieve) metadata from ffnet. So I'm wondering if a medium term strategy might be to correlate ffnet metadata with ao3 and maybe some other sites like ff2ebook, and redirect ffnet retrievals to those other sites. There are also a few crawls (not up to date) of ffnet on archive.org and maybe other places, so possibly fanficfare could retrieve from those.

Is this the right place for this discussion? I'd rather not sign up for github.
It's also more a question of data than code.