[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by AndersW

In configure FFF under custom columns. When I select the column I want prequelHTML to go to, prequelHTML isn't an option that I can choose.

Correct. Only standard metadata items (common to most/all sites) are available in the GUI.

Site specific metadata (like prequelHTML for FimF or snapeflavours for potionsandsnitches.org to name just a couple) can only be set to custom columns using custom_columns_settings in personal.ini.

That's because there are a ton of different site specific metadata items available and because users can add their own additional metadata items to do complex metadata manipulations on.

You do have to have a GUI switch turned on to use custom_columns_settings however:

[AndersW]

I think I have it working now. I am downloading some stories to check.

[bugstomper]

Originally Posted by JimmXinu

updating existing ebooks with new chapters without downloading all the old ones again is just as important as downloading whole stories.

So right now the implementation of getting new chapters is to re-read the first page of the story to get the latest list of chapter links and then download chapters starting from the next one that has not yet been downloaded?

What about when there is no such list, including the original links in the metadata of the ebook, or just the link to the last chapter that was downloaded, so nothing has to be downloaded again?

[JimmXinu]

Originally Posted by bugstomper

So right now the implementation of getting new chapters is to re-read the first page of the story to get the latest list of chapter links and then download chapters starting from the next one that has not yet been downloaded?

As part of gathering story metadata in doExtractChapterUrlsAndMetadata(), the list of chapters (URLs and names) is retrieved and saved. Whether that's collected from the first chapter or its own page is site dependent.

FFF does a simple comparison of number of chapters in epub vs gathered when determining whether to update using "Update EPUB if new chapters".

While actually doing the download & update, FFF looks at the the collected chapter URLs and compares each to the already-downloaded chapters inside the saved epub (each chapter saved it's URL in a hidden tag). That way any chapter, regardless of where in the list, is only downloaded if it's not already present.

Originally Posted by bugstomper

What about when there is no such list, including the original links in the metadata of the ebook, or just the link to the last chapter that was downloaded, so nothing has to be downloaded again?

As I said, I don't think we support any sites that don't have a chapter list. Certainly none of the adapters I wrote.

If all the chapters are already present (IE, 'Update EPUB Always'), the existing downloaded chapters are all reused.

I suggest going to PM or email if you want continue this discussion--it's getting a bit esoteric.

[Ser4nb2LUY6e]

Originally Posted by gcomyn

I've been working on it, but I am having trouble with the cookies... I've never been able to figure them out...

zero experience with python, but with a little bit of google-fu I came up with this:

C:\Program Files\Calibre2>calibre-debug.exe
Welcome to the interactive calibre shell!

>>> import urllib2
>>> req = urllib2.Request('https://www.webnovel.com/book/7618111306000905')
>>> res = urllib2.urlopen(req)
>>> cookie = res.info()['Set-Cookie']
>>> cookie
'_csrfToken=kIBXdGijBxz13gkj9SweYHmtWOht0q9xUOp0K0BF; expires=Tue, 12-Jun-2018 03:18:22 GMT; path=/; domain=.webnovel.com; secure'
>>>

[JimmXinu]

New test version posted

2017-06-12
- Tweak skip_threadmarks_categories comments in INI.
- Special error msg for storiesonline.net about Listing Theme.
- Change adapter_fanfiktionde to https.
- Extend base_xenforoforum_adapter Reader Mode to other Threadmark Categories.
- Update translations.
- Correct an error log message.

Known outstanding issues:
- FimF user/pass -- FaceDeer is looking at it.
- webnovel.com is broken -- GComyn is looking at it.

[Ser4nb2LUY6e]

Originally Posted by JimmXinu

- webnovel.com is broken -- GComyn is looking at it.

Here is my attempt at fixing the webnovel adapter. First time I did anything in python, so someone should look closely at what I did. It works for me fine (as far as I can tell) for both the initial download of a new books as well as update of books originally downloaded with the original adapter (before the site changes broke it).

[JimmXinu]

New test version posted

2017-06-13
- Update adapter_webnovelcom for site changes. Thanks, Ser4nb2LUY6e

Thanks, I did make changes to fit into the existing system, but your code was very clean and concise.

[Ser4nb2LUY6e]

Originally Posted by JimmXinu

Thanks, I did make changes to fit into the existing system, but your code was very clean and concise.

You are welcome. Thanks for fixing it up. I clearly don't know enough yet about the specific internals of fanficfare and I just figured out the "right" code using the calibre-debug interactive console and some google-fu...

Edit: An initial quick test seems to show that the test plugin as released is working well.

[JimmXinu]

New test version posted

2017-06-14
- Fix base_xenforoforum_adapter for QQ--it doesn't have threadmark categories or reader mode.
- Fix fimfiction datePublished (Merge pull request #190 from FaceDeer/master)
- http->https for adapter_midnightwhispers

[taratears]

Quick question: Can fff be used to crawl sites (such as fanfiction.net) looking for stories that fit certain criteria?

[JimmXinu]

Originally Posted by taratears

Quick question: Can fff be used to crawl sites (such as fanfiction.net) looking for stories that fit certain criteria?

No.

It's for downloading stories you already know about to ebooks for ease of use on ebook readers and for archiving.

[taratears]

Originally Posted by JimmXinu

No.

It's for downloading stories you already know about to ebooks for ease of use on ebook readers and for archiving.

Thank you very much.

[JimmXinu]

New test version posted

2017-06-15
- Add keep_prequel_in_description option for fimfiction.net.

(AndersW asked for it and I forgot about it for a while.)

[Atherton25]

Originally Posted by taratears

Quick question: Can fff be used to crawl sites (such as fanfiction.net) looking for stories that fit certain criteria?

Scryer is an excellent search tool for extensive filtering down the ff.net crap. While it's not exactly what you're looking for, it does a much better job of getting to the stories you want.

Anyway, I thought I'd throw that in. Just in case you didn't know about it.