[GUI Plugin] FanFicFare

[Keybounce]

If you are unable to update Firefox, are you also on an older version of Calibre? It does make a difference.

Yeah, actually. Calibre 3.48, used in both OS's, so I keep the base Calibre the same and update the plug-ins.


One of the OS's could go to calibre 6, but then I can't use the same book database (other OS can't use that.)

[astarinlife]

Hi, I've been trying my best to figure this out myself but I'm stumped. I'm trying to get any character name that I don't have a replace_metadata entry for to just display the first name in ships. I found a post that was close earlier in the thread and I believe that this bit is what I'm looking for

sort_ships:true
ships_CHARS=>^([^ ]+).*$=>\1

but when those are working none of other ships entrys work anymore. I go from Naruto/Hinata to Uzumaki/Hyuuga so I know it works! Just not as I would like it to. Please help? Here's my whole code, if that helps.

## This is an example of what your personal configuration might look
## like.  Uncomment options by removing the '#' in front of them.

[defaults]
browser_cache_path:C:\Users\astar\AppData\Local\Microsoft\Edge\User Data\Default\Cache
browser_cache_path:C:\Users\astar\AppData\Local\Google\Chrome\User Data\Default\Cache\Cache_Data
browser_cache_age_limit:4.0
is_adult:true
strip_chapter_numbers:true
add_chapter_numbers:true
add_genre_when_multi_category: Crossover

[epub]
include_images:true
keep_summary_html:true

[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true
always_login:true
exclude_notes:authorheadnotes,chaptersummary,chapterheadnotes,chapterfootnotes,authorfootnotes,inspiredlinks
strip_chapter_numbers:true

extra_valid_entries:fandoms, freeformtags, freefromtags,
 ao3categories, comments, chapterslashtotal, chapterstotal, kudos,
 hits, bookmarks, collections, byline, series00,
 series01, series02, series03, series00Url, series01Url, series02Url,
 series03Url, series00HTML, series01HTML, series02HTML, series03HTML,
 ships2chars,onlyromships
fandoms_label:Fandoms
freeformtags_label:Freeform Tags
ao3categories_label:AO3 Categories
comments_label:Comments
chapterslashtotal_label:Chapters/Total Chapters
chapterstotal_label:Total Chapters
kudos_label:Kudos
hits_label:Hits
collections_label:Collections
bookmarks_label:Bookmarks

series00HTML_label:Series
series01HTML_label:Additional Series
series02HTML_label:Additional Series
series03HTML_label:Additional Series

make_linkhtml_entries:series00,series01,series02,series03

include_in_chapterstotal:chapterslashtotal.NOREPL
add_to_replace_metadata:
 chapterstotal=>^([0-9])+/(.+)$=>\2

keep_in_order_freeformtags:true
keep_in_order_ao3categories:true
keep_in_order_fandoms:true
keep_in_order_genre:true
keep_in_order_category:true
keep_in_order_characters:true
keep_in_order_ships:true

include_in_genre: freeformtags, ao3categories

include_in_freefromtags:freeformtags

titlepage_entries: seriesHTML,rating,warnings,ao3categories,fandoms,
 ships,characters,freeformtags,status,datePublished,dateUpdated,
 numWords,chapterslashtotal,kudos,hits,bookmarks,series01HTML,series02HTML,series03HTML,
 site,description

extra_subject_tags:fandoms,freeformtags,ao3categories,onlyromships

include_in_onlyromships:ships

add_to_include_metadata_post:
 onlyromships=~[ ]*/[ ]*

include_in_ships2chars:ships.NOREPL

add_to_replace_metadata:
 ships2chars=>[ ]*/[ ]*=>/
 ships2chars=>[ ]*&[ ]*=>/
 ships2chars=>([^/]+)/([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3\,\4
 ships2chars=>([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3
 ships2chars=>([^/]+)/(.+)=>\1\,\2

include_in_characters:characters,ships2chars

add_to_custom_columns_settings:	
 chapterslashtotal=>#chapters
 freeformtags=>#freeformtags
 onlyromships=>#ships
 description=>#description
 bookmarked=>#bookmarked
 kudos=>#kudos
 hits=>#hits
 fandoms=>#fandoms
 series01=>#series01
 series02=>#series02

extra_logpage_entries:fandoms,freeformtags,ao3categories,comments,
 chapterslashtotal,kudos,hits,bookmarks,collections,byline,restricted,series00,
 series01,series02,series03
sort_ships:true
replace_metadata: 
 fandoms=> \(.*\)=>
 fandoms=>(.*)\1-.*=>\1
#Marvel
 fandoms=>Agent Carter.*=>Avengers
 fandoms=>[Aa]nt-[Mm]an.*=>Avengers
 fandoms=>[Cc]aptain [Aa]merica.*=>Avengers
 fandoms=>[Cc]aptain [Mm]arvel.*=>Avengers
 fandoms=>[Dd]octor [Ss]trange.*=>Avengers
 fandoms=>[Ii]ron [Mm]an.*=>Avengers
 fandoms=>The Avengers.*=>Avengers
 fandoms=>Fantastic Four=>
 fandoms=>[Xx]-[Mm]en.*=>
 fandoms=>[Mm]arvel.*=>Avengers
 fandoms=>Thor.*=>Avengers
 fandoms=>.*Winter Soldier.*=>Avengers
 fandoms=>Daredevil.*=>Avengers
#Spideypool
 fandoms=>Cable and Deadpool=>
 fandoms=>[Dd]eadpool.*=>Deadpool
 fandoms=>.*[Ss]pider-[Mm]an.*=>Spider-Man
#Magicians
 fandoms=>Queliot - Fandom=>
 fandoms=>[Tt]he [Mm]agicians.*=>The Magicians
#Harry Potter
 fandoms=>[Hh]arry [Pp]otter.*=>Harry Potter
#BHA
 fandoms=>僕のヒーローアカデミア \| Boku no Hero Academia \| My Hero Academia=>Boku no Hero Academia
#Witcher
 fandoms=>Wiedźmin.*=>The Witcher
#Teen Wolf
 fandoms=>[Tt]een [Ww]olf.*=>Teen Wolf
#Fast and the Furious
 fandoms=>.*Furious.*=>Fast and the Furious
#Other
 fandoms=>RPF=>
 fandoms=>Compilation of =>
 fandoms=>.*- Fandom=>
 fandoms=>.*[Nn]aruto.*=>Naruto
 fandoms=>Fruits Basket.*=>Fruits Basket
 fandoms=>[Dd]ark-[Hh]unter.*=>Dark-Hunter
 fandoms=>[Cc]hronicles [Oo]f [Nn]arnia.*=>Chronicles of Narnia
 fandoms=>.*Final Fantasy.*=>Kingdom Hearts
 fandoms=>His Dark Materials.*=>His Dark Materials
 fandoms=>DCU=>
## Fix and standardize your character names first
 ships=> \(.*\)=>
 ships=>Original Female Character\(s\)=>OFC
 ships=>Original Male Character\(s\)=>OMC
 ships=>Original Character\(s\)=>OC
#BHA
 ships=>Aizawa Shouta \| Eraserhead=>Aizawa
 ships=>Amajiki Tamaki=>Tamaki
 ships=>Asui Tsuyu=>Tsuyu
 ships=>Bakugou Masaru=>Masaru
 ships=>Bakugou Mitsuki=>Mitsuki
 ships=>Bubaigawara Jin \| Twice=>Twice
 ships=>Chisaki Kai \| Overhaul=>Overhaul
 ships=>Fukukado Emi \| Ms. Joke=>Ms. Joke
 ships=>Hakamata Tsunagu \| Best Jeanist=>Best Jeanist
 ships=>Hikiishi Kenji \| Magne=>Magne
 ships=>Iguchi Shuuichi \| Spinner=>Spinner
 ships=>Iida Tensei \| Ingenium=>Ingenium
 ships=>Iida Tensei=>Ingenium
 ships=>Inui Ryou \| Hound Dog=>Hound Dog
 ships=>Izumi Kouta=>Kouta
 ships=>Kaminari Denki=>Denki
 ships=>Kayama Nemuri \| Midnight=>Midnight
 ships=>Midoriya Hisashi=>Hisashi
 ships=>Midoriya Inko=>Inko
 ships=>Monoma Neito=>Monoma
 ships=>Sako Atsuhiro \| Mr. Compress=>Mr. Compress
 ships=>Sasaki Mirai \| Sir Nighteye=>Sir Nighteye
 ships=>Sensei \| All For One=>All For One
 ships=>Shimura Nana=>Nana
 ships=>Shuuzenji Chiyo \| Recovery Girl=>Recovery Girl
 ships=>Sousaki Shino \| Mandalay=>Mandalay
 ships=>Takami Keigo \| Hawks=>Hawks
 ships=>Takeyama Yuu \| Mount Lady=>Mount Lady
 ships=>Todoroki Enji \| Endeavor=>Endeavor
 ships=>Todoroki Fuyumi=>Fuyumi
 ships=>Todoroki Natsuo=>Natsuo
 ships=>Todoroki Rei=>Rei
 ships=>Todoroki Touya=>Dabi
 ships=>Toogata Mirio=>Mirio
 ships=>Tsuchikawa Ryuuko \| Pixie-bob=>Pixie-bob
 ships=>Usagiyama Rumi \| Miruko=>Miruko
 ships=>Utsushimi Camie=>Camie
 ships=>Yagi Toshinori \| All Might=>All Might
 ships=>Yamada Hizashi \| Present Mic=>Present Mic
#Fast and the Furious
 ships=>Dominic Toretto=>Dom
#Harry Potter
 ships=>Albus Dumbledore=>Dumbledore
 ships=>Fenrir Greyback=>Greyback
 ships=>Gellert Grindelwald=>Grindelwald
 ships=>Gregory Goyle=>Goyle
 ships=>Minerva McGonagall=>McGonagall
 ships=>Nymphadora Tonks=>Tonks
 ships=>Severus Snape=>Snape
 ships=>Theodore Nott=>Nott
#Inuyasha
 ships=>Higurashi Kagome=>Kagome
#Kingdom Hearts
 ships=>Leon \(Kingdom Hearts\)\/Squall Leonhart\/Cloud Strife=>
 ships=>Squall Leonhart=>Leon
#Naruto
 ships=>Haruno Sakura=>Sakura
 ships=>Hatake Kakashi=>Kakashi
 ships=>Hatake Sakumo=>Sakumo
 ships=>Hoshigaki Kisame=>Kisame
 ships=>Hyuuga Hinata=>Hinata
 ships=>Hyuuga Neji=>Neji
 ships=>Katou Dan=>Dan
 ships=>Kyuubi \| Nine-tails \| Kurama=>Kurama
 ships=>Maito Gai \| Might Guy=>Guy
 ships=>Namiashi Raidou=>Raidou
 ships=>Namikaze Minato=>Minato
 ships=>Nara Shikamaru=>Shikamaru
 ships=>Nohara Rin=>Rin
 ships=>Shimura Danzo=>Danzo
 ships=>Shiranui Genma=>Genma
 ships=>Uchiha Fugaku=>Fugaku
 ships=>Uchiha Itachi=>Itachi
 ships=>Uchiha Mikoto=>Mikoto
 ships=>Uchiha Obito=>Obito
 ships=>Uchiha Sasuke=>Sasuke
 ships=>Uchiha Shisui=>Shisui
 ships=>Umino Iruka=>Iruka
 ships=>Uzumaki Kushina=>Kushina
 ships=>Uzumaki Naruto=>Naruto
 ships=>Yamanaka Ino=>Ino
 ships=>Yamato \| Tenzou=>Tenzou
#Queer as Folk
 ships=>Melanie Marcus=>Mel
#Magicians
 ships=>William "Penny" Adiyodi=>Penny
#Marvel
 ships=>Ellie Phimister=>Negasonic
 ships=>Frigga \| Freyja=>Freyja
 ships=>James "Bucky" Barnes=>Bucky
 ships=>James "Rhodey" Rhodes=>Rhodey
 ships=>Mary Jane Watson=>MJ
 ships=>Michelle Jones=>MJ
 ships=>Phil Coulson=>Coulson
 ships=>Stephen Strange=>Strange
#Soul Eater
 ships=>Franken Stein=>Stein
 ships=>Nakatsukasa Tsubaki=>Tsubaki
#Suicide Squad
 ships=>Harleen Quinzel=>Harley Quinn
#Teen Wolf
 ships=>Jordan Parrish=>Parrish
 ships=>Vernon Boyd=>Boyd
#Witcher
 ships=>Cirilla Fiona Elen Riannon=>Ciri
#Other
 ships_CHARS=>^[pP]ast (.*)=>00past\,\1
 ships_CHARS=>^([^ ]+).*$=>\1
 ships=>^00past\s*(/|&)\s*=>Past\s


[dark-solace.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true
exclude_notes:authorheadnotes,chaptersummary,chapterheadnotes,chapterfootnotes,authorfootnotes,inspiredlinks

[www.fanfiction.net]
never_make_cover: true
exclude_notes:authorheadnotes,chaptersummary,chapterheadnotes,chapterfootnotes,authorfootnotes,inspiredlinks
strip_chapter_numbers:true
use_browser_cache:true
use_browser_cache_only:true

[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
#collect_series: true

[JimmXinu]

Originally Posted by astarinlife

I go from Naruto/Hinata to Uzumaki/Hyuuga so I know it works! Just not as I would like it to. Please help? Here's my whole code, if that helps.

When ever asking for help, it's best to include a few story URLs that exhibit the problem, otherwise we're guessing.

That said, my guess is you bumped into a story where the auther put the names in Japanese order: "Uzumaki Naruto/Hyuuga Hinata".

And ships_CHARS replacements are applied before ships replacements. So ships_CHARS=>^([^ ]+).*$=>\1 is happening before ships=>Uzumaki Naruto=>Naruto, thus "Uzumaki".

Changing ships=>Uzumaki Naruto=>Naruto to ships_CHARS=>Uzumaki Naruto=>Naruto (and other lines like it) may work.

I don't see a way around this as a general case--you want 'first name' AKA 'given name', but when 'given name' isn't always first, you're kinda stuck.

Also, I'll tell you the same thing I tell everyone trying to normalize AO3 metadata: it's going to be a lot of work and it's never going to be perfect as long as AO3 allows authors to enter whatever they want.

If AO3 had a mode that displayed the 'canonical' tags instead we could at least leverage the work already done by AO3 'tag wranglers' team. (And no, looking up each tag to find the canonical version isn't feasible; authors routinely put tens or even hundreds of tags on a story.)

[astarinlife]

Thank you so much for all the help and tips!

Changing ships=>Uzumaki Naruto=>Naruto to ships_CHARS=>Uzumaki Naruto=>Naruto (and other lines like it) may work.

This 100% solves my problem.

I'm very much going into this with a Work Hard! Once. mindset. After the initial slog I can just add a few things here and there with new fandoms hopefully.

Lastly, are there any resources available for me to try and learn a bit more about how to work with this code? I had no idea that ships_CHARS was processed first, and I would like to learn how this all works, if I can. I'm very new at this. I just did some regex tutorials the other day. Again, thank you for any and all help.

[JimmXinu]

Originally Posted by astarinlife

Lastly, are there any resources available for me to try and learn a bit more about how to work with this code? I had no idea that ships_CHARS was processed first, and I would like to learn how this all works, if I can. I'm very new at this. I just did some regex tutorials the other day. Again, thank you for any and all help.

There are a number of pages in the FFF github wiki. Searching this thread is another good source, although MR's search is a bit limited.

Honestly, I realized many years ago that, a) I don't write very good documentation, and; b) I hate writing documentation. So I don't write very much. But it's a wiki, others can add to it also.

[MCBastos]

Ran into a bit of a problem with the anthology feature in a particular series:
https://www.tthfanfic.org/Series-42
Basically, the description page is exceptionally long and exceeded the room in the FFF window. The problem is that instead of turning into a scrollable window, apparently the windows just expanded downwards offscreen, taking the controls with it into unavailable-land. I tried expanding the window sideways in hope of reaching the end, but it wasn't enough.
Fortunately when I hit <Enter> blind, FFF went ahead and processed the series.
It's not a dealbreaker issue, but it's a bit annoying.

[JimmXinu]

New Test Version Posted

2022-08-19
- Make series name/comment area scrollable for when lengthy.

[MCBastos]

Went back to the troublesome series with the test version. It's about a thousand percent better, thank you.

[ownedbycats]

Question: If I want to exclude certain series from the Calibre metadata but retain it on the title page, is there a better option than using add_to_extra_valid_entries to duplicate series?

EDIT: Also, for this fic, I can't get this replace_metadata working:

 category=>Lilo & Stitch: The Series \(Cartoon\)=>Lilo & Stitch

The regex matches, so what am I doing wrong?

[JimmXinu]

Originally Posted by ownedbycats

Question: If I want to exclude certain series from the Calibre metadata but retain it on the title page, is there a better option than using add_to_extra_valid_entries to duplicate series?

exclude_metadata_post setting using a conditional to look for individual story urls/ids? Nothing else comes to mind that doesn't use an added entry.

Originally Posted by ownedbycats

EDIT: Also, for this fic, I can't get this replace_metadata working:

 category=>Lilo & Stitch: The Series \(Cartoon\)=>Lilo & Stitch

The regex matches, so what am I doing wrong?

Use &amp; instead of just &. This one I did document.

[Keybounce]

Originally Posted by JimmXinu

If you are unable to update Firefox, are you also on an older version of Calibre? It does make a difference.

So the problem with this site was tracked down to requiring the latest version of TLS in the browser/client code. The site admin allowed one previous version, which was enough to solve the issue.

[Amalthia]

Hi,

Is it possible to list the word count from the story metadata in the Calibre's tags if you don't have the Count Pages plugin installed or a custom column for word count?

I read through the FanFicFare default options but didn't see anything about word count for archiveofourown.org.

On AO3 the stats label says "Words:"

Thanks,
Amalthia

[JimmXinu]

Originally Posted by Amalthia

Is it possible to list the word count from the story metadata in the Calibre's tags if you don't have the Count Pages plugin installed or a custom column for word count?

Word count is one of standard metadata entries collected for most sites. The FFF entry name is numWords.

You can set it to a Calibre custom column from the FFF Custom Column config tab, or using custom_columns_settings from personal.ini.

If you literally mean what you mention above, and want it in Calibre's Tags, you can do that with personal.ini setting include_subject_tags (or add_to_include_subject_tags), but be aware that Calibre automagically will change commas ',' in the word count to semicolons ';' because Tags are comma separated.

If you want to do that, but remove the commas and/or add a label to the number, I suggest making a new extra_valid_entries in personal.ini and manipulating that. Some internal FFF features (such as Words Added in logpage) depend on numWords not being changed.

[JimmXinu]

New Test Version Posted

2022-08-21
- Remove RSS link from threadmarks_title (XF2)
- Also include threadmarks_title in tagsfromtitle (XF)
- nsapa_proxy: detect proxy protocol violation - Fix #865 - thanks, nsapa

UPDATE - added #865 fix

[Amalthia]

Originally Posted by JimmXinu

Word count is one of standard metadata entries collected for most sites. The FFF entry name is numWords.

You can set it to a Calibre custom column from the FFF Custom Column config tab, or using custom_columns_settings from personal.ini.

If you literally mean what you mention above, and want it in Calibre's Tags, you can do that with personal.ini setting include_subject_tags (or add_to_include_subject_tags), but be aware that Calibre automagically will change commas ',' in the word count to semicolons ';' because Tags are comma separated.

If you want to do that, but remove the commas and/or add a label to the number, I suggest making a new extra_valid_entries in personal.ini and manipulating that. Some internal FFF features (such as Words Added in logpage) depend on numWords not being changed.

Thanks for explaining how this would work! I tried a few different methods and each one failed. I used your "add_to_include_subject_tag" and "numWords" and the tags field was populated with the word count.

Thanks!
Amalthia