[GUI Plugin] FanFicFare

[ownedbycats]

Originally Posted by ownedbycats

Question: I have include_in_series:series,collections, and a include_metadata_pre on collections so that it only captures a few specific ones for fic swaps and the like.

I believe that pre means it captures before doing the replace_metadatas. So would it be better practice to do replace_metadata on collections itself?

After some thought: editing collections makes more sense to me. Though if I'm wrong please correct.

Originally Posted by ownedbycats

EDIT: Another question: does include_metadata_pre or exclude_metadata_pre run first?

Example: Suppose I have this, so that selected collections are included if there's no series.

include_metadata_pre:
 collections=~(Collection 1|Collection 2)&&series!~^.*$

But then I also have this, to exclude "Some Series"

exclude_metadata_pre:
 series=~Some Series

If a book has both "Collection 1" and "Some Series," what should be the intended result?

I found a fic to actually test this on. I ended up with the collection and no series, which is what I had wanted.

[oenwolf]

Originally Posted by JimmXinu

Removed NSFW link for you.

What misbehavior are you seeing? Looks like that story downloaded fine for me.

I ran calibre-debug. Traceback below.

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 721, in do_loop
    self.foreach_function(book)
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1343, in prep_download_loop
    story = self.get_story_metadata_only(adapter)
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1215, in get_story_metadata_only
    adapter.getStoryMetadataOnly(get_cover=False)
  File "/Users/partwalk/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_adapter.py", line 320, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/Users/partwalk/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/adapter_adultfanfictionorg.py", line 221, in doExtractChapterUrlsAndMetadata
    for i, chapter in enumerate(chapters.findAll('a')):
AttributeError: 'NoneType' object has no attribute 'findAll'

[JimmXinu]

Are you sure have the most recent test version? (see link in my sig) Because that's exactly what changed...

Because if you do, we'll have to try things like a special build that dumps the page contents to log and you sending the log to me off thread.

Originally Posted by oenwolf

I ran calibre-debug. Traceback below.

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 721, in do_loop
    self.foreach_function(book)
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1343, in prep_download_loop
    story = self.get_story_metadata_only(adapter)
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1215, in get_story_metadata_only
    adapter.getStoryMetadataOnly(get_cover=False)
  File "/Users/partwalk/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_adapter.py", line 320, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/Users/partwalk/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/adapter_adultfanfictionorg.py", line 221, in doExtractChapterUrlsAndMetadata
    for i, chapter in enumerate(chapters.findAll('a')):
AttributeError: 'NoneType' object has no attribute 'findAll'

[oenwolf]

Originally Posted by JimmXinu

Are you sure have the most recent test version? (see link in my sig) Because that's exactly what changed...

Because if you do, we'll have to try things like a special build that dumps the page contents to log and you sending the log to me off thread.

It's not the latest test version. I'd said it was the latest release per Github v.4.17.0 with the changes you made in September. I see now you've made further changes in October that aren't in this release. That's cool, I'll wait for the release of v.4.18.0 before trying again. Thanks for your help.

[ownedbycats]

I ran into an issue with my collection-series.

One of my fics (mature, so PM'ing link) has no series, but one collection that does not match my include_metadata:

## Copy selected collections to series
include_in_series:series,collections

include_metadata_pre:
 collections=~(N7 Exchange|N7 Exchange \d{4}|Spectre Requisitions \d{4}|Spec Recs|Spec Recs \d{4})&&series!~^.*$

Still, the collection keeps showing up in the series field. I tried both with and without the appending &&series!~^.*$

It probably doesn't matter, but I also have a never_make_cover:true for the url.

Any idea what is happening?

[JimmXinu]

Originally Posted by ownedbycats

## Copy selected collections to series
include_in_series:series,collections

include_metadata_pre:
 collections=~(N7 Exchange|N7 Exchange \d{4}|Spectre Requisitions \d{4}|Spec Recs|Spec Recs \d{4})&&series!~^.*

Still, the collection keeps showing up in the series field. I tried both with and without the appending &&series!~^.*$

Used as shown above, the include_metadata_pre line is ignored due to infinite recursion--series includes collection, but collection depends on a conditional series.

(It should appear in the job log: "Skipping include_metadata_pre key(collections) value(Fandom_Nerd123_MaleShepard_Stories) line(collections=~(N7 Exchange|N7 Exchange \d{4}|Spectre Requisitions \d{4}|Spec Recs|Spec Recs \d{4})&&series!~^.*$) to prevent infinite recursion.")

After removing &&series!~^.*, it works as expected for me blocking 'Fandom_Nerd123_MaleShepard_Stories'.

But if you already did that, it must be something else. Perhaps a different include line matching, or the setting not being in the correct section or overridden, etc.

[ownedbycats]

I tried again without the series modifier, this time using 'update always' instead of 'update Calibre metadata' and it worked. I'm not sure why that would make the difference.

[ownedbycats]

One other difference: before my last test, I had deleted the existing series value from Calibre. Is it possible that 'update metadata' could've returned a "undefined {don't overwrite existing values}" instead of a "undefined {overwrite existing values}"? I'm not even sure if it's possible.

[ownedbycats]

Question: Some AO3 authors include both "CharacterA/CharacterB" and "CharacterA & CharacterB."

Has anybody made a in/exclude_metadata regex + conditional that can strip out the later without making separate lines for every possible ship? (Is it even possible?)

[JimmXinu]

Originally Posted by ownedbycats

One other difference: before my last test, I had deleted the existing series value from Calibre. Is it possible that 'update metadata' could've returned a "undefined {don't overwrite existing values}" instead of a "undefined {overwrite existing values}"? I'm not even sure if it's possible.

Correct. FFF will not reset Series column to empty if it's already populated. Otherwise you couldn't manually use the Series column with sites that don't have series, such as ffnet.

There are subtle but important differences between the FFF metadata series and the Calibre standard column Series. I assumed you were talking about the former.

[JimmXinu]

New Version Released

Version 4.18.0 - 2022-11-21
- Update metadata caching with dependency invalidating
- Still allow images with use_flaresolverr_proxy if use_browser_cache
- Support classic AND modern (and minimalist) theme for storiesonline, finestories and scifistories - thanks, mvlcek
- adapter_tenhawkpresents: Change site to t.evancurrie.ca - tenhawk domain semi-broken
- remove_class_chapter missing from config lists
- Remove Calibre Update Cover option entirely(was deprecated) #878
- adapter_adultfanfictionorg: Fixes for site changes, thanks cryosaur.

[ownedbycats]

Originally Posted by JimmXinu

Correct. FFF will not reset Series column to empty if it's already populated. Otherwise you couldn't manually use the Series column with sites that don't have series, such as ffnet.

There are subtle but important differences between the FFF metadata series and the Calibre standard column Series. I assumed you were talking about the former.

That makes sense. I thought that mentioning 'update Calibre metadata' would've made it clear that I meant the column. (Unless it also updates somewhere else?)

[ownedbycats]

Regex question:

[https://archiveofourown.org/works/9652229]
strip_chapter_numbers: true
chapter_title_strip_pattern:(Ch[0-9]+:|Chapter [0-9]+, Part [0-9]+|Ch. [0-9]+|Ch[0-9]+)

The regex won't capture the , Part [0-9]+ bit in some of the chapter titles. Any ideas?

EDIT: Might be easier to just remove FFF's added numbering?

[ownedbycats]

Also, is it intended/known that copying from a browser into the config also copies the formatting? I was adding comments to know what fic each of the story sections is for.

EDIT: Also, what happens if there's more than one section for a story? I just noticed this happened but apparently it didn't get any errors (and somehow I didn't notice).

[JimmXinu]

Originally Posted by ownedbycats

Regex question:

[https://archiveofourown.org/works/9652229]
strip_chapter_numbers: true
chapter_title_strip_pattern:(Ch[0-9]+:|Chapter [0-9]+, Part [0-9]+|Ch. [0-9]+|Ch[0-9]+)

The regex won't capture the , Part [0-9]+ bit in some of the chapter titles. Any ideas?

EDIT: Might be easier to just remove FFF's added numbering?

You patter is looking for "Chapter #, Part #", your example on regex101.com has "Ch9, Part 2".

Here's a more general regexp that matches all your examples:

Ch(apter)?[ \.]*[0-9]+(, Part [0-9]+)?

The setting to turn off FFF chapters numbers; you have them turned on somewhere--they are off by default:

## add_chapter_numbers can be true, false or toconly
## (Note number is not added when there's only one chapter.)
add_chapter_numbers:false

Originally Posted by ownedbycats

Also, is it intended/known that copying from a browser into the config also copies the formatting? I was adding comments to know what fic each of the story sections is for.

EDIT: Also, what happens if there's more than one section for a story? I just noticed this happened but apparently it didn't get any errors (and somehow I didn't notice).

The Qt edit box accepts the formatting, but it is discarded on save. I think I vaguely recall that it showed up when text color coding was added and was too difficult to separate.

See https://github.com/JimmXinu/FanFicFa...cated-sections