[GUI Plugin] FanFicFare

[baggins41]

Originally Posted by JimmXinu

- fimfiction adapter: add new tag types, fix groups (#259), thanks FaceDeer!

Prior to this update, I'd noticed the "Human" tag wasn't showing up in metadata anymore on stories downloaded from FimFic. When I saw that line in the release notes, I'd hoped it had been fixed, but I just tried the new version on https://www.fimfiction.net/story/398801/constellations and still no dice.

It looks like the "Alternate Universe" tag is similarly being missed.

[JimmXinu]

Originally Posted by baggins41

Prior to this update, I'd noticed the "Human" tag wasn't showing up in metadata anymore on stories downloaded from FimFic. When I saw that line in the release notes, I'd hoped it had been fixed, but I just tried the new version on https://www.fimfiction.net/story/398801/constellations and still no dice.

It looks like the "Alternate Universe" tag is similarly being missed.

Those are being collected and put into the site specific metadata entry content (because that's what FimF calls them in the code). Doing it that way or including them in genre is kind of a toss-up and FaceDeer elected to make it separate.

You can include content in genre in your personal.ini with:

[www.fimfiction.net]
include_in_genre:genre,content

I can see why FaceDeer made the call to keep content separate, but if enough people think it should be included in genre (or if those tags were in genre before), I can add include_in_genre:genre,content to defaults.ini.

[baggins41]

I certainly wouldn't argue against the logic of having those tags processed separately. It seems an entirely reasonable thing to do and the option for more end-user control over how metadata is processed is always welcome.

However, given those tags were treated as genre tags in the past and that the FimFiction website displays them in the same way as it does genre tags, adding include_in_genre:genre,content would seem to be a sensible default arguing from the Principle of Least Surprise.

[JimmXinu]

I agree, especially if that's how it was before.

I also noticed that the FimF code sets fandom, but not category. By default, fandom will now be copied to category to match what most of our site adapters do.

New test version posted

2018-01-26
- Tweak FimF settings so 'genre' also contains 'content' tags and 'category' contains 'fandom' tags.

[baggins41]

Something isn't quite right with the test version's include_in_category:fandom setting. All the stories I'd downloaded from FimFic with previous versions showed "Category: My Little Pony: Friendship is Magic" on their title pages. With the new test version, Category is omitted entirely.

Just for giggles, I added include_in_category:category to my personal.ini and that resolved the issue.

Other than that, everything seems to be working as expected. Thanks!

[JimmXinu]

baggins41 - That's my mistake. I didn't realize there was an setting for extracategories:My Little Pony: Friendship is Magic. Plus, I used 'fandom' when it's really fandoms so category was coming out entirely empty.

(FYI: It's category and genre singular vs fandoms plural for most other metadata entries for 'historical reasons'.)

Changing the setting to include_in_category:category,fandoms does what I intended, but there's another wrinkle.

Most, but not all, stories have the 'content' tag 'MLP: FiM' which gets put into fandoms so now category would contain both: "MLP: FiM, My Little Pony: Friendship is Magic". (An example of a story with multiple fandoms. An example of a story without MLP: FiM in fandoms.)

That lead me to notice that many of tags have longer 'title' attributes. "MLP: FiM" has the full "My Little Pony: Friendship is Magic"; "Celestia"=>"Princess Celestia", etc (followed by a stories count that would be removed).

So here are the options that come to mind:

1. Change only "MLP: FiM" to "My Little Pony: Friendship is Magic" in fandoms for consistency with older downloads. Keep include_in_category:category,fandoms setting.
2. Change all tags (fandoms, characters, etc) to the longer title attribute version, giving consistency for category and longer, more detailed tags, but inconsistencies compared to previously collected characters, etc.
3. Same as one of the two above, but also remove extracategories:My Little Pony: Friendship is Magic setting so stories without "MLP: FiM" tag (eg "Equestria Girls") don't get "My Little Pony: Friendship is Magic" in category.
4. Remove the new include_in_category setting from defaults; category would go back to containing only "My Little Pony: Friendship is Magic" and individual users can make their own choices regarding fandoms.

I would lean towards options 1 or 4 because they offer the most consistency and the least work for me.
But I don't read FimF, so I'd like to hear options from people who do before making the choice.

[readerboy7]

Whenever I try to download a fic from FFnet or fictionpress, I get the following error:

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

(example URLs: https://www.fanfiction.net/s/9750097/1 , https://www.fictionpress.com/s/2961893/1/)

Note that the SpaceBattles, SufficientVelocity, fimficiton, and AO3 fics I tried to download worked fine. It's only FFnet and fictionpress that gave me issues.

Additionaly, I am capable of viewing these pages in my browser. Not sure if that matters.

In the event you need it, personal.ini (safe version) is in the spoiler below.

Spoiler:

## This is an example of what your personal configuration might look
## like. Uncomment options by removing the '#' in front of them.

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels. Example:

## [defaults]
## titlepage_entries: category,genre, status
## [www.whofic.com]
## # overrides defaults.
## titlepage_entries: category,genre, status,dateUpdated,rating
## [epub]
## # overrides defaults & site section
## titlepage_entries: category,genre, status,datePublished,dateUpdated,dateCreated
## [www.whofic.com:epub]
## # overrides defaults, site section & format section
## titlepage_entries: category,genre, status,datePublished
## [overrides]
## # overrides all other sections
## titlepage_entries: category

## Some sites also require the user to confirm they are adult for
## adult content. Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites? You can use strip_chapter_numbers to strip them off. Just
## want to make them all look the same? Strip them off, then add them
## back on with add_chapter_numbers. Don't like the way it strips
## numbers or adds them back? See chapter_title_strip_pattern and
## chapter_title_add_pattern.
#strip_chapter_numbers:true
add_chapter_numbers:true

## Add this to genre if there's more than one category.
add_genre_when_multi_category: Crossover

## Some sites/authors/stories use br tags instead of p tags for
## paragraphs. This feature uses some heuristics to find and replace
## br paragraphs with p tags while preserving scene breaks.
#replace_br_with_p: true

# 'add_to_' concats on to existing param, thus the leading comma
add_to_extra_valid_entries:,oneshot
# oneshot is a copy of status because status itself isn't a list.
include_in_oneshot:status
# add oneshot to the tags that are included in epub & calibre.
add_to_include_subject_tags:,oneshot

# Change oneshot to ['Completed'] to ['Completed','Oneshot'] only when
# numChapters is exactly 1.
# with ',' instead of '\,' it would be ['Completed'] to
# ['Completed,Oneshot']--one string instead of two.
add_to_replace_metadata:
oneshot=>Completed=>Completed\,Oneshot&&numChapter s=>^1$
Pokemon=>Pokémon

[epub]
## include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible. Images work
## in epub format only. To get mobi or other format with images,
## download as epub and use Calibre to convert.
#include_images:true

## If not set, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
#keep_summary_html:true

## If set, the first image found will be made the cover image. If
## keep_summary_html is true, any images in summary will be before any
## in chapters.
#make_firstimage_cover:true

## Resize images down to width, height, preserving aspect ratio.
## Nook size, with margin.
#image_max_size: 580, 725

## Change image to grayscale, if graphics library allows, to save
## space.
#grayscale_images: false


## Most common, I expect will be using this to save username/passwords
## for different sites. Here are a few examples. See defaults.ini
## for the full list.

[www.twilighted.net]
#username:XXXXXXXX
#password:XXXXXXXX
## default is false
#collect_series: true

[ficwad.com]
#username:XXXXXXXX
#password:XXXXXXXX


[www.adastrafanfic.com]
## Some sites do not require a login, but do require the user to
## confirm they are adult for adult content.
#is_adult:true

[www.fanficauthors.net]
username:XXXXXXXX
password:XXXXXXXX

[www.fictionalley.org]
#is_adult:true

[www.harrypotterfanfiction.com]
#is_adult:true

[www.fimfiction.net]
#is_adult:true
#fail_on_password:XXXXXXXX

[www.tthfanfic.org]
#is_adult:true
## tth is a little unusual--it doesn't require user/pass, but the site
## keeps track of which chapters you've read and won't send another
## update until it thinks you're up to date. This way, on download,
## it thinks you're up to date.
#username:XXXXXXXX
#password:XXXXXXXX


# I added this myself
[www.fanfiction.net]
add_to_keep_html_attrs:,style
add_to_replace_tags_with_spans:,font

[forums.spacebattles.com]
make_firstimage_cover: false


## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site. Set true here to force all sites to
## collect series.
#collect_series: true

[https://forums.spacebattles.com/threads/472801/]
make_firstimage_cover: false
default_cover_image:https://forums.spacebattles.com/data...jpg?1482945866

[https://forums.spacebattles.com/threads/347113/]
make_firstimage_cover: false
default_cover_image:https://forums.spacebattles.com/data...jpg?1454529093


[https://forums.spacebattles.com/threads/414320/]
make_firstimage_cover: false

[https://forums.spacebattles.com/threads/389744/]
make_firstimage_cover: false

[https://forums.spacebattles.com/thre...aneum.352191/]
make_firstimage_cover: false

[www.example.com]
make_firstimage_cover: false

[JimmXinu]

readerboy7 - Your problem is most likely fixed by this FAQ. We've seen it several times now.

[readerboy7]

Ah, thanks. Yeah, that fixed the problem. Sorry for not checking the FAQ.

[JimmXinu]

New Test Version Posted

2018-02-04
- Handle new VIP chapter types in adapter_webnovelcom (#263) - thanks, cryzed
- Fix FimF category/genre tags somemore. *Don't* include fandoms in category for fimfiction.net by default.

Originally Posted by JimmXinu

...
4. Remove the new include_in_category setting from defaults; category would go back to containing only "My Little Pony: Friendship is Magic" and individual users can make their own choices regarding fandoms.

I would lean towards options 1 or 4 because they offer the most consistency and the least work for me.
But I don't read FimF, so I'd like to hear options from people who do before making the choice.

Lacking any other input from FimF users, I went with option 4.

[baggins41]

Originally Posted by JimmXinu

Lacking any other input from FimF users

Sorry about that, Jimm. I've been largely unplugged from the internet for the past week.

To be honest, I personally don't have a strong feeling about the handling of fandoms in general, so in that regard, your choice works fine for me. (The whole "fandom" tag thing is a fairly new addition to FimFic in the first place and will probably only be really important once Knighty's "GenFic" project fully launches.)

I would offer that, up until recently, the longer character tags have been the default. In your example, all the stories I've downloaded for the past couple of years that included the character of Celestia have been tagged "Princess Celestia", not "Celestia".

If it's a major undertaking to use the longer tags, it's not a huge pain point for me, personally, but it would be marginally more convenient if they were consistent with the old convention.

[Tanjamuse]

Is there any way I can download metadata without downloading the story?

[JimmXinu]

Originally Posted by Tanjamuse

Is there any way I can download metadata without downloading the story?

If you already have a book in your library there's the "Update Calibre Metadata from Web Site" option. That will work even if you create an empty book and set the Identifier url: to the story URL.

The plugin doesn't have a way to download the metadata for a story without having (or creating) a book in your library at all. However, you can create basically empty books by giving a chapter range of 0: storyurl[0]. Eg: https://www.fanfiction.net/s/9598828/1/[0]

CLI does have a '-m' or '--meta-only' option that is intended for debugging purposes.

[JimmXinu]

New Test Version Posted

2018-02-14
- Update adapter_efpfanficnet to use https and remove www. by default.

2018-02-04
- Handle new VIP chapter types in adapter_webnovelcom (#263) - thanks, cryzed
- Fix FimF category/genre tags somemore. *Don't* include fandoms in category for fimfiction.net by default.

2018-01-26
- Tweak FimF settings so 'genre' also contains 'content' tags and 'category' contains 'fandom' tags.

[JimmXinu]

New test version posted

2018-02-15
- Fix rating, warnings, add ships to adapter_harrypotterfanfictioncom