[GUI Plugin] FanFicFare

[baggins41]

Originally Posted by JimmXinu

Question for Plugin Users
There are several options available on every download/update:

1. Output Format
2. Update Mode
3. Update Calibre Metadata
4. Update EPUB Cover
5. Background Metadata (updates only)

[...]

Which are these settings are actually useful today on every download/update?

Output Format: Never touch it.
Update Mode: Frequently use most of the modes.
Update Calibre Metadata: Currently on; never touch it.
Background Metadata (updates only): Currently off; never touch it.
Update EPUB Cover: The setting that, historically, has confused me the most and the number one reason I decided to respond as soon as I saw the question.

For a long while I was unclear on what "Update EPUB Cover" even did. I tried it both off and on and didn't see any difference in behavior that affected me so I ultimately ended up leaving it alone. (Nowaday's I do understand what it does and leave it on. I can't think of any reason why I'd ever want it off for a particular download.)

I customize the covers of nearly every book I download, so what I really hoped it might do when I first started experimenting with it was toggle between two behaviors:

• Replace the current Calibre cover with one downloaded or generated based on whatever is configured under the "Calibre Cover" settings tab.
• Leave the cover currently contained in the Calibre metadata completely untouched.

Obviously, I quickly learned my hopes were unfounded.

Right now, any time I'm going to update a bunch of existing books that have customized covers, I go into settings and flip "Update Calibre Cover (from EPUB)" to "No", then remember to set it back to " Yes, if EPUB..." when I'm done. It's a minor inconvenience, but an inconvenience nonetheless.

As long as you're working in cleaning up that bit of the code, I don't suppose there's any possibility of a per-download option that would quickly guarantee leaving the Calibre cover alone (wink-wink-nudge-nudge)?

[watani]

Originally Posted by watani

Seems that fanfiction.net just updated their site in a way that broke how FFF gets metadata and chapters for stories on the site. I'm getting this error for every story on FF.net I try to update with the latest test version:

'utf-8' codec can't decode byte 0x9d in position 422: invalid start byte'

This error also occurs in a way that results in no job getting listed in the Calibre log.

So this error message that I got previously has reared its ugly head again. Restarting Calibre doesn't fix the problem.

I've attached the Calibre debug log to see if that helps figure out what's wrong.

Edit: The error is definitely tied to how FFF is reading the Firefox cache. Once I disabled the setting for FFF to use the cache to update FF.net chapters, the error message went away.

[Ascello]

Is there a way to modify error messages? Not directly, I want to do feed it into new entry and modify that so my calibre error column is cleaner. The only issue is that I don't know how errors are called in .ini, since the obv "error" does not work.

[Rei97]

So I noticed that when calibre with fanficfare plugin gathers tags meta data, it orders them in alphabetic order and not in the order it is on ao3. I don't know if its the default plugin behaviour or I did something but is there any way to order tags in original order instead? This is very important especially for my custom relationship category, as some writers order the relationship name tags in order of importance to the story ( for instance first pairing tag is likely to be the main while the pairings that follow are likely to be less important side pairing). But if it orders tags in alphabetical order, I no longer have a clue, which pairing is likely to be the main.
Also I heard that in theory , placing this into personal.ini should work :

keep_in_order_characters:true

keep_in_order_ships:true

keep_in_order_freeformtags:true

keep_in_order_ao3categories:true

keep_in_order_genre:true

but it does not seem to work;;

Please help figure out how to make it work or any other solution you may have

[ownedbycats]

Did you put it under the [archiveofourown.org] section? I do and it works:

[archiveofourown.org]
## Preserve tag order
keep_in_order_freeformtags:true
keep_in_order_ao3categories:true
keep_in_order_genre:true

[JimmXinu]

Multi-reply

Originally Posted by ownedbycats

Question: using conditionals, is it possible to search for a phrase/phrases in the story description and do a replace_metadata based on them?

You cannot use a conditional for the same metadata entry you are trying to modify--that way lies madness infinite recursion.

But you shouldn't need to, either. It's a pattern match and replace. If it doesn't match, it doesn't replace.

If you want something more complex, we've talked before about making a copy of an entry with .NOREPL. That can be used for conditional.

Originally Posted by watani

So this error message that I got previously has reared its ugly head again. Restarting Calibre doesn't fix the problem.

I've attached the Calibre debug log to see if that helps figure out what's wrong.

I'm not able to replicate that problem and without seeing it myself to experiment with it, I'm unlikely to be able to fix it.

Are you still needing to use browser cache? Because ffnet's been working fine for me with just cloudscraper (on by default).

Originally Posted by Ascello

Is there a way to modify error messages? Not directly, I want to do feed it into new entry and modify that so my calibre error column is cleaner. The only issue is that I don't know how errors are called in .ini, since the obv "error" does not work.

Not in FFF. You could make a custom Calibre column that re-formats/edits a hidden column containing FFF's errors.

Originally Posted by Rei97

So I noticed that when calibre with fanficfare plugin gathers tags meta data, it orders them in alphabetic order and not in the order it is on ao3. I don't know if its the default plugin behaviour or I did something but is there any way to order tags in original order instead?
...

Except for author, FFF automatically sorts list type metadata entries.

keep_in_order_* is the correct way to get FFF to not sort them.

However, as I recall, Calibre will order tags type columns even if FFF doesn't. If entries are in the 'correct' order you want in the title page, but are sorted in the Calibre column, you need to change the type of the Calibre column.

[Rei97]

Originally Posted by ownedbycats

Did you put it under the [archiveofourown.org] section? I do and it works:

[archiveofourown.org]
## Preserve tag order
keep_in_order_freeformtags:true
keep_in_order_ao3categories:true
keep_in_order_genre:true

yes I did, and I tested again and yes and these 3 rules seem to work , only "keep_in_order_ships:true" doesnt seem to work for my relationship colimn column and thats the most important one for me. While it displays ships in metadata column log in correct order, in my custom relationship column it still displays relationships in alphabetical order

[Rei97]

Originally Posted by ownedbycats

Did you put it under the [archiveofourown.org] section? I do and it works:

[archiveofourown.org]
## Preserve tag order
keep_in_order_freeformtags:true
keep_in_order_ao3categories:true
keep_in_order_genre:true

Originally Posted by Rei97

yes I did, and I tested again and yes and these 3 rules seem to work , only "keep_in_order_ships:true" doesnt seem to work for my relationship colimn column and thats the most important one for me. While it displays ships in metadata column log in correct order, in my custom relationship column it still displays relationships in alphabetical order

Edit: I just fixed it, it was the problem with my custom column-I didnt click "contains names" in the tag section. when i checked that box, it started ordering correctly. Thank your for your help!

[Rei97]

Originally Posted by ownedbycats

Did you put it under the [archiveofourown.org] section? I do and it works:

[archiveofourown.org]
## Preserve tag order
keep_in_order_freeformtags:true
keep_in_order_ao3categories:true
keep_in_order_genre:true

Please ignore my last message
I just fixed it, it was the problem with my custom column-I didnt click "contains names" in the tag section. when i checked that box, it started ordering correctly

[watani]

Originally Posted by JimmXinu

I'm not able to replicate that problem and without seeing it myself to experiment with it, I'm unlikely to be able to fix it.

Are you still needing to use browser cache? Because ffnet's been working fine for me with just cloudscraper (on by default).

Strictly speaking, no I don't need to use the browser cache. I was just wondering if there was some way to catch that error and proceed to use cloudscraper instead of abruptly quitting the whole update process (i.e. what already happens by default if FFF can't find the new chapter(s) in cache). That way I don't have to completely disable the backup method of reading from Firefox cache. But if it's impossible to implement without replicating it on your end, then it's fine if you leave things as is.

[JimmXinu]

Originally Posted by watani

Strictly speaking, no I don't need to use the browser cache. I was just wondering if there was some way to catch that error and proceed to use cloudscraper instead of abruptly quitting the whole update process (i.e. what already happens by default if FFF can't find the new chapter(s) in cache). That way I don't have to completely disable the backup method of reading from Firefox cache. But if it's impossible to implement without replicating it on your end, then it's fine if you leave things as is.

That particular error is happening while FFF is initially reading the cache index. When it fails, the entire cache fails and no pages will be able to be read from the cache.

In that situation, I think it's better to fail entirely.

[thenico]

Hello,

I don't know if anyone use the nsapa_proxy feature but I pushed an update for the proxy.
CloudFare have updated their anti-bots and they now detect Firefox piloted by Selenium (the technology that nsapa_proxy used) so infinite captcha page.

Since Mozilla is not helping (see the git change for technical detail), I had to switch to Chrome (at least version 90).

It is probably still broken somewhere (CloudFare is currently not sending me to the captcha page so this function is untested).

Best regards,
NS

[booster-t]

Hi all:

Well, looks like Cloudflare is at it again.. everything worked fine yesterday. This morning, nope.

Here's the key info from the debug file:

FFF: INFO: 2021-04-29 09:31:52,590: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.base_adapter(257): continue_on_chapter_error: (https://www.fanfiction.net/s/13822562/2/) cloudscraper reports: "Cloudflare IUAM possibility malformed, issue extracing delay value."

Using Calibre 5.16.1 and plugin 4.10 on Mac 11.3. I am using the cache and even paged through the whole story to see if that was a solution.

Stephen

[Ascello]

I can confirm the newest issue with FF.net, since I got same error. This had to happen today, as yesterday everything worked fine.

[ownedbycats]

Personally unable to confirm myself on FFF 4.1.0 and Calibre running from source. If of any difference, my slow_down_sleep_time is set to 15.