[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by cecabre

... When I downloaded a new story, the new pairing column filled in right away. But I can't figure out how to get the old fics to update the pairings. I was sure it had something do with the "Updating existing fanfiction books" setting, but I can't get it to work.
...

FanFicFare only updates columns when you download or update. There are several modes for updating existing ebooks. It sounds like you should:

1. Select the existing books,
2. Click the FanFicFare button (or Update Existing FanFiction Books from the FanFicFare drop down),
3. Click 'Show Download Details' at the bottom left,
4. Change 'Update Mode' to 'Update Calibre Metadata from Website',
5. Click OK.

If you plan to continue adding columns, rather than hitting the site again and again, I suggest setting up the Saved Metadata Column feature and doing 'Update Calibre Metadata from Website' once after that.

Then there's another 'Update Mode', 'Update Calibre Metadata from Saved Metadata Column' that you can use that will be much faster and not hit the site.

Originally Posted by JimmXinu

...
Near the bottom of the Custom Column tab in FanFicFare config, look for Saved Metadata Column. Can only be used with 'Long Text' custom columns.

I figure the primary use case for this is people who are making changes to their Calibre columns and replace_metadata. Rather than hitting the site servers repeatedly as they update and test, they can update from data already in the library. (I had previously proposed putting this data inside the epub, but that metadata could be unintentionally removed by Calibre's Polish Book or other epub editing tools.)
...

[cecabre]

Thank you for the help! I had actually tried the 'Update Calibre Metadata from Website' mode before and hadn't been able to get it to work, but this time around I figured out that it wasn't updating because it required confirmation of adult status. So I figured out how to add my AO3 details so it would stop that.

I will definitely set up that saved metadata option. I'm still figuring out what columns I need, so it sounds like that will be very helpful. Edit: found what I was looking for! And I set up the saved metadata, then made a new column to test it out and it worked! Thanks again!

[Tanjamuse]

Could the "Get Story URLs from Selected Books" be made to work with formats other than epub? Since FFF can download stories as both mobi, epub, html and txt I had expected it would work with at least those formats as well.

Thanks in advance

[JimmXinu]

Originally Posted by Tanjamuse

Could the "Get Story URLs from Selected Books" be made to work with formats other than epub? Since FFF can download stories as both mobi, epub, html and txt I had expected it would work with at least those formats as well.

I'll look into searching html and txt formats when present.

I'm not interested in searching mobi. It's an outdated, proprietary, binary format.

[Tanjamuse]

Thanks for considering this. I hope it's possible.

[JimmXinu]

New test version posted


2017-05-09
- Clear extratags: for tgstorytime.com and fictionmania.tv.
- Fix error with std_cols_newonly if user has never saved config.
- Adapt to masseffect2.in redesign - thanks PlushBeaver
- Let PI search inside zip(html) and txt formats for story URLs.

[Tanjamuse]

I just tested with a txt-file and it works just fine.

Thank so much for the speedy work.

[JimmXinu]

New test version posted

Performance improvement of Rejects List reading so it doesn't take so long the first time every time you restart Calibre or change libraries.

2017-05-11
- Reorder sections in defaults.ini files alphabetically, save script that does it.
- Remove a debug output from adapter_fanficauthorsnet
- One-off normalize Reject List URLs to save doing it all the time. Also, automatically save plugin version in prefs.
- Update translations. (For: Let PI search inside zip(html) and txt formats for story URLs.)

[Tanjamuse]

The stories from StoriesofArda.com doesn't pick up the status tag.

Even though the status is complete it's marked in calibre as incomplete.

Here's a link: http://www.storiesofarda.com/chapterlistview.asp?SID=2

[JimmXinu]

New test version posted

2017-05-15
- Change fanficauthors.net to https.
- Catch other common complete/in-progress statuses for calibre yes/no columns.
- adapter_wwwaneroticstorycom: 'Complete'->'Completed'
- Update Translations.

Unless something urgent comes up, this is going to be the last test version before the next release.

[JimmXinu]

New Version Released:

Version 2.11.0 - 17 May 2017
- Change fanficauthors.net to https.
- Catch other common complete/in-progress statuses for calibre yes/no columns.
- adapter_wwwaneroticstorycom: 'Complete'->'Completed'
- Reorder sections in defaults.ini files alphabetically, save script that does it.
- Remove a debug output from adapter_fanficauthorsnet
- One-off normalize Reject List URLs to save doing it all the time. Also, automatically save plugin version in prefs.
- Update translations. (For: Let PI search inside zip(html) and txt formats for story URLs.)
- Clear extratags: for tgstorytime.com and fictionmania.tv.
- Fix error with std_cols_newonly if user has never saved config.
- Adapt to masseffect2.in redesign - thanks PlushBeaver
- Let PI search inside zip(html) and txt formats for story URLs.
- Fix AO3 use_view_full_work feature for 1 chapter works.
- New site alternatehistory.com as a base_xenforoforum_adapter. Plus base_xenforoforum_adapter fixes.
- Add site specific extracategories to new sites.
- Fix for PI suppressauthorsort/suppresstitlesort interacting wrong with Author/Title New Only. (PI only)
- Fix CLI -f option help for text vs txt.
- Add AO3 feature use_view_full_work -- true by default.
- Update included dependency packages
- Update html5lib from 0.9x7 to 0.9x9.
- Update to BeautifulSoup 4.5.3.
- Update six.py to 1.10.0.
- Update chardet to 3.0.2.
- Update html2text to (2016, 9, 19).
- Add chardet_confidence_limit option for 'auto' encoding setting.
- Add webencodings to included_dependencies. Needed by in web service with newer html5lib.
- Don't include html5lib(and webencodings) and six in PI zip--uses calibre's versions anyway.
- New sites from GComyn:
- Add calibre_series_meta optional feature to include series metadata like calibre in epubs. For CLI & Web Service.
- Updated date added to webnovel by cryzed
- AO3 site-specific metadata bookmarktags bookmarksummary and option always_login added.
- Fixes for adapter_gravitytalescom from GComyn.
- New sites from GComyn:
- inkbunny.net
- lcfanfic.com
- noveltrove.com
- gravitytales.com
- imrightbehindyou.com
- mtt.just-once.net
- obidala.net
- sword.borderline-angel.com
- tasteofpoison.inkubation.net
- unknowableroom.org
- www.13hours.org
- www.aneroticstory.com
- www.deandamage.com
- www.fiction.thebrokenworld.org
- www.narutofic.org
- www.starskyhutcharchive.net
- www.thedelphicexpanse.com
- www.thundercatsfans.org
- www.webnovel.com
- Add Story Notes to base_efiction_adapter.
- Don't send Referer:None -- hpfanficarchive.com doesn't like it.
- Additional tags collected for wuxiaworld--thanks GComyn
- Add download.archiveofourown.org for AO3 due to old downloaded AO3 epubs
- Fix for author including Updated in desc in adapter_ashwindersycophanthexcom.
- Fix "got more than 100 headers"-issue adapter_royalroadl.py only.
- Fix ffnet referer for cover images.
- Fix for a mistaken Norwegian Bokmål translation.
- Fix ffnet referer for cover images.

[MCBastos]

One note and one question...

The note is regarding the SWI (Stories With Images) at Fictionmania. Here's the thing: FM has several reading modes, which correspond to variant URLs.
The standard URL for text-only stories (the one FFF recognizes) is in this format:
http://www.fictionmania.tv/stories/readtextstory.html?storyID=<story ID number>
This will show the story in plaintext format embedded into a HTML page which supplies metadata. I don't disagree that it's the best choice for FFF.
There are also, however, variant URLs in these formats:

http://www.fictionmania.tv/stories/readststory.html?storyID=<story ID number>
(shows the story in raw plaintext format, with no metadata)

javascript:newPopwin('/stories/readxstory.html?storyID=<story ID number>')
(pops up a windows with the plaintext story automatically converted to HTML)

...and that's it for the plaintext ones. However, FM ALSO allows stories pre-formatted in HTML, usually including images. These will have URLs in the format:
https://www.fictionmania.tv/stories/readhtmlstory.html?storyID=<story ID number>
instead of the links to the "plaintext embedded in HTML" and "story autoconverted to HTML" as above.

Here's the thing: FFF recognizes ONLY the first of these URL formats, the one with "readtextstory". Which is not a big deal for plaintext stories... but IS a problem with SWIs, because FM does not expose a link in this format for those. However, I experimented editing manually the link before invoking it from FFF, and FFF IS capable of downloading those stories (albeit in a plaintext-with-no-images fashion).

Now, I'm not really requesting support for downloading the HTML-with-images version, because I have no idea how much work it would entail (might be trivial, might be a big job...). But it would be nice if FFF would at least accept the "readhtmlstory" variant, even if it actually downloaded the "readtextstory" version. At least we would get the metadata, which is a chore to do by hand.

Now, the question, which is unrelated...

I happen to follow a few authors who post in multiple sites -- under different pen names (sometimes due to site rules; for instance, TTH does not allow spaces in author names...).
Is there a way to configure FFF (or maybe Calibre itself) to automatically replace the variant pen name with the preferred form at download time?

[JimmXinu]

Originally Posted by MCBastos

The note is regarding the SWI (Stories With Images) at Fictionmania.
...

@cryzed wrote the adapter for fictionmania.tv, so hopefully he'll be willing to look into that. We can PM him in a couple days if he doesn't notice this.

Originally Posted by MCBastos

I happen to follow a few authors who post in multiple sites -- under different pen names (sometimes due to site rules; for instance, TTH does not allow spaces in author names...).
Is there a way to configure FFF (or maybe Calibre itself) to automatically replace the variant pen name with the preferred form at download time?

Yes, the replace_metadata feature in personal.ini. There's replace_metadata and personal.ini documentation, plus it's been discussed in detail here in the forum many times for different issues.

A couple examples from my own personal.ini:

[www.fanfiction.net]
add_to_replace_metadata:
 author=>Bobmin356=>Bobmin

[www.siye.co.uk]
add_to_replace_metadata:
 author=>_kb_=>kb0

I will warn you, however, if you have 'Set Calibre Author URL' checked in the FFF Standard Columns config tab, when you download stories from different sites with the same author name, the Calibre author URL will only remember the most recent. And Calibre will also consider it a change to all books by that author when the URL does change.

[Atherton25]

@JimmXinu
Just want to note a possible future issue, something I've run into with the beta. It's not a huge problem, but something you may want to look into.

I am using the latest beta, 2.99.5, and I'm also using the update of FFF made yesterday if it matters.

Here's the (possible) issue, and a quick explanation as to how I ran into it. First, I have a list of ~60 fics that I follow that update almost daily between webnovel/royalroad/gravity tales/etc. Of these, Royal Road is the only one that has some sort of email updates and there's only a couple of fics on there and they do update daily.

So, due to the regular updates I have assigned a 'group' to them to make them easy to check, highlighting them all and running an update. Generally within a 24 hour period (I check them every evening) 65-70% have updated. Checking as a group saves me a lot of time and works well since they update so often.

Previously when I did this, those that had an update would take 1-2 seconds apiece to 'update' while the remaining would update immediately, the remaining 15-20 books taking 1-2 seconds as a whole. Now though, every story takes about the same amount of time, even those that didn't update.

I did one check, turning off the 'Last Checked' column, and everything went back normal speeds, but when it was enabled I got the slow down.

I'm not 100% sure what's caused the slow down, only that I started running the beta recently and I had updated to the most current FFF almost at the same time. I do know this wasn't the case about a week ago when I was paying attention, and FFF was almost current maybe 2 'betas' back. I believe there's some interaction in the new beta causing the problem, I'm just not sure what it could be, so I thought I'd let you know about it and let you investigate if you wanted to.

[JimmXinu]

Originally Posted by Atherton25

...
Previously when I did this, those that had an update would take 1-2 seconds apiece to 'update' while the remaining would update immediately, the remaining 15-20 books taking 1-2 seconds as a whole. Now though, every story takes about the same amount of time, even those that didn't update.

I did one check, turning off the 'Last Checked' column, and everything went back normal speeds, but when it was enabled I got the slow down.
...

For the moment, I'm going to chalk that up to performance differences with the Calibre 2.99 beta series. The 'Last Checked' feature means you are updating all those books in the Caliber library DB, so it makes sense at least.

If you see it using current Calibre version, I'd like to know about it.