[GUI Plugin] FanFicFare

[applegaa]

Originally Posted by ownedbycats

Update if New Chapters checks for updates by comparing the number of chapters. You can use Update ePub always, which will also update the title page.

I'm not worried about the title page. I just want to update the metadata in Calibre. Does "Update EPUB Always" also do that? I guess I should test that. Thanks.

- Andrew

[JimmXinu]

Originally Posted by applegaa

I'm not worried about the title page. I just want to update the metadata in Calibre. Does "Update EPUB Always" also do that? I guess I should test that. Thanks.

If you look at the drop list of Update Modes, one is "Update Calibre Metadata from Web Site". It sounds like that's what you want.

There's a wiki page specifically detailing the UpdateModes.

[dailyalice]

Is there a way to do the equivalent of "Set 'Series [0]' for New Anthologies" for custom series columns? (Though actually my problem has more to do with the series name in anthologies. If they aren't set with a series index of 0, that's fine, I'd settle for a number 1 or something, it's the series name that's the biggest problem.)

The problem I'm running into is with the AO3 series columns I've set up (#series1, #series2, #series3, and so on) to capture additional series info. For anthologies I build from series on AO3, they wind up messy. See attached images which include a screenshot of both Book Details and Edit Metadata showing the issue.

The series name winds up being the series name over and over with the series index in the mix I guess. Instead of a series named "we miss being ruffians" in this example, for series 1 it's a series name of "we miss being ruffians [1], we miss being ruffians [2], we miss being ruffians"

The series type I'm using for those columns is "Text column for keeping series-like information."

I've set up the AO3 series columns in personal.ini with series00 filling my #series1 column, series01 filling #series2, and so on. I've tried every possible mode by now: r, a, n, r_anthaver, and n_anthaver.

I have "take value from first source book" set for the #series1, #series2, #series3, & #series4 columns in ePubMerge though I think I've tried last book before too.

(The series I'm using in this example: https://archiveofourown.org/series/770673 though I've run into the issue with lots of others. I've been just manually correcting the series name and changing the index to 0 but I thought I'd ask if there was a better way.)

Maybe I'm missing something obvious (that'd be nice, even though I hate wasting people's time).

[JimmXinu]

Originally Posted by dailyalice

The series name winds up being the series name over and over with the series index in the mix I guess. Instead of a series named "we miss being ruffians" in this example, for series 1 it's a series name of "we miss being ruffians [1], we miss being ruffians [2], we miss being ruffians"
...
I've set up the AO3 series columns in personal.ini with series00 filling my #series1 column, series01 filling #series2, and so on. I've tried every possible mode by now: r, a, n, r_anthaver, and n_anthaver.

My first thought is that custom_columns_settings mode 'r' should take care of it.

However, it might be coming from your other personal.ini settings, can I see your personal.ini? The safe version, of course.

[dailyalice]

Originally Posted by JimmXinu

My first thought is that custom_columns_settings mode 'r' should take care of it.

However, it might be coming from your other personal.ini settings, can I see your personal.ini? The safe version, of course.

Thanks.

I stripped my personal.ini way down a time or two and still had the issue. Attached is one of the shorter test versions where I still have the issue.

[beaRL]

Hii,

Sorry to bother you again.
I've a little issue with two Yes/No custom columns after I install the new update for both Fanficfare (v4.17.0) and Calibre (v6.7.1.)

One column used to show if I'd Bookmarked a story in Ao3 and in the other I manually indicate if I've read a story. After updating, the result of the Bookmarked column is duplicated in the read column.



I've been checking default.ini in case there's something I need to remove or change from my personal.ini but I can't figure it out.

## This is an example of what your personal configuration might look
## like.  Uncomment options by removing the '#' in front of them.

[defaults]

## extra tags (comma separated) to include, primarily for epub.
extratags: FanFiction

## As a work around for certain sites blocking automated downloads,
## FFF now offers the ability to look for pages in your Chrome(or
## Chromium-derived) or Firefox browser's cache.  Requires both
use_browser_cache:true

## browser_cache_path needs to be set to the location of YOUR browser
## cache.  Here are a few examples for different OS.  Note that all
## have YOUR user name in them somewhere as well as a Profile name,
## frequently Default.  Make sure you have personalized (and
## uncommented) ONE browser_cache_path setting.  Also note that Chrome
## browsers added an additional directory level, 'Cache_Data', in
## early 2022.

## Note also that browser_cache_path goes under [defaults] because
## browser cache is read once and shared between all downloads for
## performance.

## Further note that there is a time cost to first reading the browser
## cache on each download session (not Calibre session)

## windows:
### Chrome:
#browser_cache_path:C:\Users\YourUser\AppData\Local\Google\Chrome\User Data\Default\Cache\Cache_Data
#browser_cache_path:C:\Users\YourUser\AppData\Local\Google\Chrome\User Data\Profile 1\Cache\Cache_Data
### Firefox
#browser_cache_path:C:\Users\YourUser\AppData\Local\Mozilla\Firefox\Profiles\ZjwI7Fo4.default\cache2

## mac:
### Chrome:
#browser_cache_path:/Users/your.user/Library/Caches/Google/Chrome/Default/Cache/Cache_Data
#browser_cache_path:/Users/your.user/Library/Caches/Google/Chrome/Profile 2/Cache/Cache_Data
### Firefox
#browser_cache_path:/Users/your.user/Library/Caches/Firefox/Profiles/43fkezvc.default-release/cache2

## linux:
### Chrome:
#browser_cache_path:/home/youruser/.cache/google-chrome/Default/Cache/Cache_Data
#browser_cache_path:/home/youruser/.cache/google-chrome/Profile 1/Cache/Cache_Data
### Firefox
#browser_cache_path:/home/youruser/.cache/mozilla/firefox/dk4o1y83.default-release/cache2

browser_cache_path:C:\Users\Beatriz\AppData\Local\Google\Chrome\User Data\Default\Cache\Cache_Data

## It's common for browser cached files to be kept for several hours
## or more.  You can limit the age of cached files FFF will use from
## browser_cache_path with browser_cache_age_limit.  Only cached files
## that were downloaded within 'browser_cache_age_limit' hours will be
## used.  If set to -1, all cached files will be used.
browser_cache_age_limit:-1

## As a (second) work around for certain sites blocking automated
## downloads, FFF offers the ability to request pages through nsapa's
## fanfictionnet_ff_proxy and FlareSolverr proxy servers.  See
## https://github.com/JimmXinu/FanFicFare/wiki/ProxyFeatures for more
## details.

## FlareSolverr (https://github.com/FlareSolverr/FlareSolverr) is a
## generic proxy that works with several otherwise blocked sites.
## It's recommended to only set use_flaresolverr_proxy:true for
## specific sites.
## FlareSolverr v1 doesn't work with some sites anymore (including
## ffnet), but FlareSolverr v2+ cannot download images.
## use_flaresolverr_proxy:true assumes FSv2 and automatically sets
## include_images:false
## If you want to use FSv1 with images, you can set
## use_flaresolverr_proxy:withimages

#[www.fanfiction.net]


## If you have the Generate Cover plugin installed, you can use the
## generate_cover_settings parameter to intelligently decide which GC
## setting to run.  There are three parts 1) a template of which
## metadata part(s) to look at, 2) a regular expression to match the
## template, and 3) the name of the GC setting to use, which must
## match exactly.  Use this parameter in [defaults], or by site eg,
## [ficwad.com]
## Make sure to keep at least one space at the start of each line and
## to escape % to %%, if used.
## template => regexp to match => GC Setting to use.
## To use this, make sure you go to the Generate Cover tab in FanFicFare
## config and check 'Allow generate_cover_settings from personal.ini
## to override'
## You can check for an existing cover image in the metadata
## cover_image.  cover_image can be:
## specific -- The site has an image that is specifically this story's cover
## first -- The first image in the story or story desc is used as cover
## default -- A default_cover_image was used
## force -- A force_cover_image was used
## old -- There was already a cover image in the epub.
## This is useful because GC plugin can use the existing image.
#generate_cover_settings:
# ${category} => Buffy:? [tT]he Vampire Slayer => BuffyCover
# ${category} => Star Trek => StarTrekCover
# ${cover_image} => ^$ => CompleteCover
# ${cover_image} => (specific|first|default|force) => CoverWithOrigImage

## [defaults] section applies to all formats and sites but may be
## overridden at several levels.  Example:
make_firstimage_cover:true

dateUpdated_label:Updated
include_in_lastupdated:dateUpdated

# 'add_to_' concats on to existing param, thus the leading comma
add_to_extra_valid_entries:,oneshot,lastupdated
# oneshot is a copy of status because status itself isn't a list.
include_in_oneshot:status

# Change oneshot from 'Completed' to 'Oneshot' only when
# numChapters is exactly 1.
 add_to_replace_metadata: oneshot=>Completed=>Oneshot&&#chapters=>^1$

add_to_custom_columns_settings:
 oneshot=>#status
 lastupdated=>#lastupdate

keep_in_order_tags: true
include_tocpage: false
collect_series: true
extra_valid_entries:kudos
kudos_label: kudos

custom_columns_settings:
 kudos=>#kudos


# create a couple new metadata entries
add_to_extra_valid_entries:,romships,platships

# label them (used with titlepage and logpage)
romships_label:Romantic Relationships
platships_label:Platonic Relationships

# copy standard entry ships into them
include_in_romships:ships
include_in_platships:ships

add_to_include_metadata_pre:
 romships=="choni"
 romships=="Lopaz"
 romships=="cheronica"
 romships=="verchoni"

add_to_exclude_metadata_pre:
 category=="choni"
 category=="Lopaz"
 category=="cheronica"
 category=="verchoni"
 category=="Buffy/Faith"
 
# only include / ships in romships and & ships in platships.
# & because HTML
add_to_include_metadata_post:
 romships=~/
 platships=~&

# optional: add romships and platships to the titlepage 
# inside the ebook.

# add_to_titlepage_entries:,romships,platships


# Set romships and platships to your custom columns.
# change #romshipscol and #platshipscol to match the 
# lookup names for your custom columns
add_to_custom_columns_settings:
 romships=>#romshipscol
 platships=>#platshipscol

# don't sort ships, romships or platships
keep_in_order_ships:true
keep_in_order_romships:true
keep_in_order_platships:true

## [defaults]
## titlepage_entries: category,genre, status
## [www.whofic.com]
## # overrides defaults.
## titlepage_entries: category,genre, status,dateUpdated,rating
## [epub]
## # overrides defaults & site section
## titlepage_entries: category,genre, status,datePublished,dateUpdated,dateCreated
## [www.whofic.com:epub]
## # overrides defaults, site section & format section
## titlepage_entries: category,genre, status,datePublished
## [overrides]
## # overrides all other sections
## titlepage_entries: category

## Some sites also require the user to confirm they are adult for
## adult content.  Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites?  You can use strip_chapter_numbers to strip them off.  Just
## want to make them all look the same?  Strip them off, then add them
## back on with add_chapter_numbers.  Don't like the way it strips
## numbers or adds them back?  See chapter_title_strip_pattern and
## chapter_title_add_pattern.
#strip_chapter_numbers:true
#add_chapter_numbers:true

## Add this to genre if there's more than one category.
#add_category_when_multi_category: Crossover

[txt]
keep_summary_html:false

## Add URLs since there aren't links.
titlepage_entries: series,seriesUrl,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,publisher,storyUrl, authorUrl, description

## Width to word wrap text output.  0 indicates no wrapping.
wrap_width: 78

## use \r\n for line endings, the windows convention.  text output only.
windows_eol: true

[epub]
## include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible.  Images work
## in epub format only.  To get mobi or other format with images,
## download as epub and use Calibre to convert.
include_images:true

## Quality level to use when converting images to jpg. Range is 0-100,
## reasonable values likely to be in the range 70-95.
jpg_quality: 95

## If not set, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
keep_summary_html:true

## If set, the first image found will be made the cover image.  If
## keep_summary_html is true, any images in summary will be before any
## in chapters.
make_firstimage_cover:true

## Resize images down to width, height, preserving aspect ratio.
## Nook size, with margin.
#image_max_size: 580, 725

## Change image to grayscale, if graphics library allows, to save
## space.
#grayscale_images: false


## Most common, I expect will be using this to save username/passwords
## for different sites.  Here are a few examples.  See defaults.ini
## for the full list.

[html] 
include_images:true
make_firstimage_cover:true
image_max_size: 580, 725 
jpg_quality: 95

output_css:

 .CI {
     margin-top:0px;
     margin-bottom:0px;
     padding:2px;
     }

 body { 
     margin:70px;
     } 

keep_summary_html:true
keep_html_attrs: true
add_to_keep_html_attrs:,style,title,align, font
keep_html_attrs:href,name,class,id,colspan,rowspan,data-orighref
anthology_tags:Anthology
sort_ships:false

remove_transparency: false



[archiveofourown.org] 
use_basic_cache:true 
## Some sites require login (or login for some rated stories) The 
## program can prompt you, or you can save it in config. In 
## commandline version, this should go in your personal.ini, not 
## defaults.ini. 
username:XXXXXXXX
password:XXXXXXXX
is_adult:true
use_view_full_work:true
extra_valid_entries:fandoms, freeformtags, freefromtags,
 ao3categories, comments, chapterslashtotal, chapterstotal, kudos,
 hits, bookmarks, collections, byline, bookmarked, bookmarktags,
 bookmarksummary, bookmarkprivate, bookmarkrec, restricted, series00,
 series01, series02, series03, series00Url, series01Url, series02Url,
 series03Url, series00HTML, series01HTML, series02HTML, series03HTML, series04HTML, series05HTML, series06HTML, series07HTML, series08HTML, series09HTML
fandoms_label:Fandoms
freeformtags_label:Freeform Tags
freefromtags_label:Freeform Tags
ao3categories_label:AO3 Categories
comments_label:Comments
chapterslashtotal_label:Chapters/Total Chapters
chapterstotal_label:Total Chapters
kudos_label:Kudos
hits_label:Hits
collections_label:Collections


## Count of bookmarks on story by all users 
bookmarks_label:Bookmarks 
## Tags & Summary from *your* bookmark on the story.  Only collected 
always_login:true 
bookmarked_label:I Bookmarked Story 
bookmarktags_label:My Bookmark Tags 
bookmarksummary_label:My Bookmark Summary 
bookmarkprivate_label:My Bookmark Private 
bookmarkrec_label:My Bookmark Rec 
restricted_label:Restricted to Registered Users 
series00HTML_label:Series 
series01HTML_label:Additional Series 
series02HTML_label:Additional Series 
series03HTML_label:Additional Series 
series04HTML_label:Additional Series 
series05HTML_label:Additional Series 
series06HTML_label:Additional Series 
series07HTML_label:Additional Series 
series08HTML_label:Additional Series 
series09HTML_label:Additional Series  
storyUrl_label:Story URL
author_label:Author
authorUrl_label:Author URL
make_linkhtml_entries:series00,series01,series02,series03,series04,series05,series06,series07,series08,series09  
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,site,storyUrl,authorUrl,description 
extra_titlepage_entries: chapterslashtotal,series01HTML,series02HTML,series03HTML,series04HTML,series05HTML,series06HTML,series07HTML,series08HTML,series09HTML,byline  

include_in_category:fandoms 

keep_in_order_category:true 
keep_in_order_genre:true 
keep_in_order_fandoms:true 
keep_in_order_freeformtags:true 
keep_in_order_ao3categories:true
keep_in_order_ships: true
keep_in_order_characters: true
keep_in_order_tags: true


add_to_custom_columns_settings:
 kudos=>#kudos
 bookmarked=>#bookmarked
 category=>#fandom

#no_image_processing: true


[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
collect_series: true

Thank you in advance!!

[JimmXinu]

Using that personal.ini, with a #series1 column, and the stories from the series URL above, it's behaving as expected for me: #series1 gets replaced with "we miss being ruffians [X]". I've tried updating a couple different ways and don't see it concatenating.

I'd be looking for issues with extra [sections] in personal.ini, making sure you are testing what you think you testing, things like that.

[dailyalice]

Originally Posted by JimmXinu

Using that personal.ini, with a #series1 column, and the stories from the series URL above, it's behaving as expected for me: #series1 gets replaced with "we miss being ruffians [X]". I've tried updating a couple different ways and don't see it concatenating.

I'd be looking for issues with extra [sections] in personal.ini, making sure you are testing what you think you testing, things like that.

Huh. Weird. Because I have the problem using that exact personal.ini. Thanks for looking into it.

Just to be clear: Series1 works correctly for individual stories in the series, turning up as story 1 of we miss being ruffians, story 2, and so on. ("we miss being ruffians [1]" and "we miss being ruffians [2]" etc.) It's just when I download the work as an anthology that Series1 is messed up.

[beaRL]

Originally Posted by JimmXinu

Using that personal.ini, with a #series1 column, and the stories from the series URL above, it's behaving as expected for me: #series1 gets replaced with "we miss being ruffians [X]". I've tried updating a couple different ways and don't see it concatenating.

I'd be looking for issues with extra [sections] in personal.ini, making sure you are testing what you think you testing, things like that.

Yes, I also have a series column and it doesn't concatenate, it only happens with the Yes/No columns. I don't know why.

Anyway I've been checking and I've just seen that the problem seems to come from a calibre update as uninstalling to a previous version solves it so I'll keep looking. Still, thank you so much for trying.

[JimmXinu]

Originally Posted by beaRL

Sorry to bother you again.
I've a little issue with two Yes/No custom columns after I install the new update for both Fanficfare (v4.17.0) and Calibre (v6.7.1.)

One column used to show if I'd Bookmarked a story in Ao3 and in the other I manually indicate if I've read a story. After updating, the result of the Bookmarked column is duplicated in the read column.

Nothing has changed in FFF that affects custom column setting in the last couple version at least.

I think you're seeing a bug with Calibre (or Qt), not FFF.

1. From your pictured settings, the #readyn column should only show icon, but your screen shot shows icon and text.
2. While your screen shot shows 'Yes' for #readyn, the icon is for 'No'.
3. I setup two columns with the same settings, populating them with an FFF download similarly shows 'Yes' with the 'No' icon even though column type is 'Icon' only. Issue confirmed.
4. I can get the same behavior manually setting #bookmarked to 'Yes'.
5. Hiding the #bookmarked column while still showing the #readyn column causes all rows to show 'Yes' with the 'No' icon for me.
6. In all cases above, the Edit metadata dialog shows the #readyn column as 'No'.

This suggest to me that it's an issue with the display code, not the value being set. Especially since a quick test with Cal6.2.1 (portable--just a version already had installed) does not show this.

I suggest you start thread in the Calibre forum or open a bug in the Calibre bug tracker. Feel free to link to this post in your explanation.

FYI, your personal.ini has extra (add_to_)custom_columns_settings that you may not realize are being applied to [defaults] because you have [www.fanfiction.net] commented out--which makes everything until the next section apply to the section above. It doesn't look like it will cause problems with your current setup, but it might some day.

[ownedbycats]

Originally Posted by beaRL

Hii,

Sorry to bother you again.
I've a little issue with two Yes/No custom columns after I install the new update for both Fanficfare (v4.17.0) and Calibre (v6.7.1.)

One column used to show if I'd Bookmarked a story in Ao3 and in the other I manually indicate if I've read a story. After updating, the result of the Bookmarked column is duplicated in the read column.

That's a Calibre bug (#1993051), already fixed in the source code. It'll be fixed next update.

Thankfully, it's just a visual bug. If it had affected data Kovid would've likely put out another minor update.

[JimmXinu]

Originally Posted by dailyalice

Just to be clear: Series1 works correctly for individual stories in the series, turning up as story 1 of we miss being ruffians, story 2, and so on. ("we miss being ruffians [1]" and "we miss being ruffians [2]" etc.) It's just when I download the work as an anthology that Series1 is messed up.

I confess, by the time I got to the end of the message, I forgot you were talking about an anthology. Now I see what you are talking about.

Honestly, it's working as designed. The seriesXX metadata entries aren't given all the same special treatment that the standard metadata series is.

I suggest using series instead of series00 to set your #series1 column, that should at least reduce the instances.

[JimmXinu]

Originally Posted by ownedbycats

That's a Calibre bug (#1993051), already fixed in the source code. It'll be fixed next update.

Thankfully, it's just a visual bug. If it had affected data Kovid would've likely put out another minor update.

[applegaa]

Originally Posted by JimmXinu

If you look at the drop list of Update Modes, one is "Update Calibre Metadata from Web Site". It sounds like that's what you want.

There's a wiki page specifically detailing the UpdateModes.

Yes and no. What I want is that when I check for updates, it will update the status if it has changed. Sure, I can then hit all of the non-updated stories with a second FanFicFare run using the "Update Calibre Metadata from Web Site" but I have my library broken down into groups to spread out my update checking and I don't want to hit each story twice in a row.

- Andrew

[ownedbycats]

Originally Posted by applegaa

Yes and no. What I want is that when I check for updates, it will update the status if it has changed. Sure, I can then hit all of the non-updated stories with a second FanFicFare run using the "Update Calibre Metadata from Web Site" but I have my library broken down into groups to spread out my update checking and I don't want to hit each story twice in a row.

- Andrew

If you use "Update Always," it will get new chapters where they exist and get the updated statuses where they do not.