[GUI Plugin] FanFicFare

[Tanjamuse]

Do the label need to be Caps locked?

[JimmXinu]

Originally Posted by Tanjamuse

Do the label need to be Caps locked?

Use the same case as the label as far as I know. INBOX is a special case.

[Tanjamuse]

thanks

[Tanjamuse]

I've tested with the new test-version and it works fine Thank you so much for the help and for a fantastic plugin.

[JimmXinu]

Another new test version (direct link)

2015-12-08

- Update to BeautifulSoup 4.4.1. (Some adapters still ues BS3)
- Add some additional BG log output.
- Suppress some debug output for the common failedtoload image case.
- Strip <script> tags from HTML. Added for FimF, but applies to all sites.

[jennifer_voorhee]

I downloaded some stories from http://tokra.fandomnet.com/ yesterday and saw that the chapters don't separate in the downloaded epub like they do from other sites, instead merging them all into "one" chapter.

On the "contents" page of the epub (where the title, author, genre, rating, chapters, etc. are all listed), the number of chapters in the story is accurate, as is everything else, so this merging of the chapters into one appears to be the only problem. I'm hopeful that this is something that can be fixed in a future update, and I can re-download these stories.

Here's a sample story for you to look at: http://tokra.fandomnet.com/viewstory...t=ok&warning=4

[JimmXinu]

Originally Posted by jennifer_voorhee

I downloaded some stories from http://tokra.fandomnet.com/ yesterday and saw that the chapters don't separate in the downloaded epub like they do from other sites, instead merging them all into "one" chapter.
...

I tried that story with EPUB format, and I didn't see any problem.

The FanFicFare mobi output doesn't correctly include chapter markings--I recommend downloading to epub and converting to azw3 for Kindle readers--that's what I do.

If you've been experimenting with chapter_title_strip_pattern, it's possible you may have stripped out the chapter titles without meaning to.

Those are the only two ideas I have that meet your described problem.

[JimmXinu]

New test version (direct link)

2015-12-09

- Remove BeautifulSoup 3, change everything to BS4. Effected parts:
- MOBI generation
- replace_br_with_p feature
- base_efiction adapters
- Most site adapters.

This is a big change that it touches over 100 files--only a few of the newer/most active adapters were already using BS4 and it isn't a drop-in replacement.

I've already spent a couple days finding and fixing bugs, but I expect there will be more. Even more than usual, I'd appreciate everyone's help testing your favorite sites with this version.

And it appears that the site sg1-heliopolis.com is offline. Anyone know anything about it?

[jennifer_voorhee]

Originally Posted by JimmXinu

I tried that story with EPUB format, and I didn't see any problem.

The FanFicFare mobi output doesn't correctly include chapter markings--I recommend downloading to epub and converting to azw3 for Kindle readers--that's what I do.

If you've been experimenting with chapter_title_strip_pattern, it's possible you may have stripped out the chapter titles without meaning to.

Those are the only two ideas I have that meet your described problem.

I just tried re-downloading that one but still have the same problem on my end. I always download in epub format. I haven't been experimenting with chapter_title_strip_pattern, at least as far as I know, but I looked through the FanFicFare configuration anyway, however don't see anything that I would think might be causing it. I've attached the epub I just downloaded for you to look at if you want. If there might be something in either FFF's or Calibre's settings that can fix it, please let me know what I should do. Thanks.

I want to reiterate that the number of chapters is correct on the "title page" of the epub, it's only when you go to the table of contents in Calibre's ebook viewer, or go to navigate to a specific chapter in an e-reader that you can easily see that the ebook is not divided into chapters.

[edeniz]

Originally Posted by jennifer_voorhee

I want to reiterate that the number of chapters is correct on the "title page" of the epub, it's only when you go to the table of contents in Calibre's ebook viewer, or go to navigate to a specific chapter in an e-reader that you can easily see that the ebook is not divided into chapters.

If I may butt in:

I had a similar problem. It turned out that the story in question did not have chapter titles, and because I'd used an aggressive stripping pattern, it looked like the story weren't divided into chapters in the epub TOC. The solution was to simply use this:

Spoiler:

strip_chapter_numbers:true

chapter_title_strip_pattern:^([0-9]+[\.: -]+)?(Chapter *[0-9]+[\.:, -]*)?

add_chapter_numbers:true

I tried downloading your link, and had no problems. Even using only the default plus the above mentioned.

[JimmXinu]

Interesting. If I go back to the last released version, I do see that problem occurring without using strip_chapter_numbers.

But in the latest test version (direct link), which is what I'm usually using, it's working fine.

So please try the test version. It's updated to use a newer HTML parser.

[jennifer_voorhee]

Originally Posted by JimmXinu

Interesting. If I go back to the last released version, I do see that problem occurring without using strip_chapter_numbers.

But in the latest test version (direct link), which is what I'm usually using, it's working fine.

So please try the test version. It's updated to use a newer HTML parser.

Thanks! I tried with the test version and now it's perfect!

Originally Posted by edeniz

If I may butt in:

I had a similar problem. It turned out that the story in question did not have chapter titles, and because I'd used an aggressive stripping pattern, it looked like the story weren't divided into chapters in the epub TOC. The solution was to simply use this:

Spoiler:

strip_chapter_numbers:true

chapter_title_strip_pattern:^([0-9]+[\.: -]+)?(Chapter *[0-9]+[\.:, -]*)?

add_chapter_numbers:true

I tried downloading your link, and had no problems. Even using only the default plus the above mentioned.

Thanks for the info. I've never made any changes to the plugin-defaults.ini before, happy to keep all the default configurations. This issue was resolved by installing the test version of FanFicFare, but I'll keep this info in mind if this problem happens again.

Once again, thank you both for your help!

[DocBrown13]

Possible bug, since downloading the latest test version I have been unable to download any stories from http://hpfanficarchive.com.

It just seems to 'hang' at the 'Downloading metadata for stories' part at the start without progressing at all no matter how long i wait and eventually I have to cancel out.

[edeniz]

I have a problem with tthfanfic.org. It has an annoying habit of putting tags like "xy-Centered" and pairing and genre (such as Romance, Action/Adventure) into "category", so that I have these things in my fandom column after download. Naturally, I'd like to move them from there to "pairings" and "genre" and "character", but try as I might, I could not come up with a working solution. Any ideas on what to do?

Another question: In fanficfare, when I'm, for example, renaming a tag like "characters=>Harry P\." I can make it "characters=>Harry P\.=>Harry Potter&&category=>Harry Potter" to make sure that change occurs only in that fandom and nowhere else. So what I'm wondering is why something similar won't work if I were to use, say, genre or pairing instead of category. Like, "rename this tag only if genre is this". (I should add that this is not terribly important, just something I'd been wondering when it didn't work when I tried it out of curiosity on a link where it seemed like it could help. I've since decided to just use replace_metadata to split one list item into multiple list entries.)

Lastly: Apparently I've screwed up something in my personal.ini without realizing, and now I get tag duplications because I renamed several tags to one. Because I'm using calibre plugin, I'd always looked at the book details browser after the download to check if all is okay, so it's now only that I'm noticing the problem. I've been going spare trying to hunt down the source the last few days, but can't find it. The problem is somewhere in this part here:

Spoiler:

[overrides]
dateCreated_format:%%Y-%%m-%%d
datePublished_format:%%Y-%%m-%%d
dateUpdated_format:%%Y-%%m-%%d

is_adult:true

[defaults]

titlepage_entries: storyId,seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,site,description

website_encodings: auto, utf8, Windows-1252
include_subject_tags: extratags, genre, category, characters, ships, status
add_to_include_subject_tags:,rating

extratags: Fanfiction

add_genre_when_multi_category: Crossover

default_value_status:Unknown
default_value_category:Fanfiction
default_value_rating:Not Rated

slow_down_sleep_time:4
connect_timeout:60.0

replace_hr: false
replace_br_with_p: true

keep_summary_html:true

keep_style_attr: true

keep_title_attr: true

title_chapter_range_pattern:${title} (Ch ${first}-${last})

strip_chapter_numbers:true

chapter_title_strip_pattern:^([0-9]+[\.: -]+)?(Chapter *[0-9]+[\.:, -]*)?

add_chapter_numbers:true


add_to_extra_valid_entries:,ships2chars
include_in_ships2chars:ships.NOREPL

add_to_replace_metadata:
 ships2chars=>[ ]*/[ ]*=>/
 ships2chars=>[ ]*&[ ]*=>/
 ships2chars=>([^/]+)/([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3\,\4
 ships2chars=>([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3
 ships2chars=>([^/]+)/(.+)=>\1\,\2

include_in_characters:characters,ships2chars

replace_metadata:
 genre=>^Unrepentant fluff!universe$|^where everything is rainbows and nothing hurts$|^More gratuitous fluff stories$|^And yet somehow fluff still works its way in argh$|^Gratuitous Fluff Story \(Not Really Meant to Be Taken Seriously\)$|^Self-Indulgent Ficcage Fluff$=>Fluff
 genre=>^All good and positive things in this fic$=>Fluff


[epub]

output_filename: ${title} by ${author} -${siteabbrev}${formatext}


titlepage_use_table: true

wide_titlepage_entries: storyUrl, authorUrl, seriesUrl



[archiveofourown.org]

do_update_hook:false

extra_valid_entries:fandoms,freeformtags,freefromtags,ao3categories,comments,kudos,hits,bookmarks,collections,series00,series01,series02,series03,series00Url,series01Url,series02Url,series03Url
fandoms_label:Fandoms
freeformtags_label:Freeform Tags
freefromtags_label:Freeform Tags
ao3categories_label:AO3 Categories
comments_label:Comments
kudos_label:Kudos
hits_label:Hits
collections_label:Collections
bookmarks_label:Bookmarks
series00_label:Series
series00Url_label:Series URL
series01_label:Series
series01Url_label:Series URL
series02_label:Series
series02Url_label:Series URL
series03_label:Series
series03Url_label:Series URL

add_to_extra_valid_entries:,series00HTML,series01HTML,series02HTML,series03HTML
series00HTML_label:Series
series01HTML_label:Additional Series
series02HTML_label:Additional Series
series03HTML_label:Additional Series

make_linkhtml_entries:series00,series01,series02,series03

titlepage_entries: storyId,seriesHTML,series01HTML,series02HTML,series03HTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,site,description

keep_in_order_genre:true 
keep_in_order_freeformtags:true 

slow_down_sleep_time:4

Here are two example links:
http://archiveofourown.org/works/719270 (-->duplicate "Crossover")
http://archiveofourown.org/works/1805503 (--->triple "Fluff")


I'd be very grateful for any and all help.

[ILB]

Originally Posted by JimmXinu

New test version (direct link)

2015-12-09

- Remove BeautifulSoup 3, change everything to BS4. Effected parts:
- MOBI generation
- replace_br_with_p feature
- base_efiction adapters
- Most site adapters.

This is a big change that it touches over 100 files--only a few of the newer/most active adapters were already using BS4 and it isn't a drop-in replacement.

I've already spent a couple days finding and fixing bugs, but I expect there will be more. Even more than usual, I'd appreciate everyone's help testing your favorite sites with this version.

I tried a few links from fanfiction.net and archiveofourown.org and downloaded epubs.
So far everything worked and the epubs look ok.