[GUI Plugin] FanFicFare

[nexusflamehart]

Originally Posted by JimmXinu

Please describe "hasn't been working for me".

Because I was able to download it. (I am using the most recent test version, tho.)

Note that it does have 4.5 million words, 1,003 chapters and 951 images.

Which is why it took 2 hours and 47 minutes to download.

Download for me took 7.5 hours and was unsuccessful.

I'll copy the error messages once I get home, but in the meantime I would be greatful for the Epub that you successfully created

[JimmXinu]

Originally Posted by nexusflamehart

Download for me took 7.5 hours and was unsuccessful.

I'll copy the error messages once I get home, but in the meantime I would be greatful for the Epub that you successfully created

You're lucky--I hadn't deleted yet. I'll PM you a link once it finishes uploading to Google Drive.

You can greatly reduce the size of the download by using a chapter range on the URL, use skip_threadmarks_categories to skip some categories such as 'Extra', or skip images with include_images:false

[madame_newbie]

Hi, is anyone facing problems downloading from Spacebattles.

I got : 'The read operation timed out' in the log. I have tried two stories and it gave me the same issue. I heard SB had some DNS issues and want to know if it's just me or is someone else also facing this problem.

Thanks.

[JimmXinu]

Originally Posted by madame_newbie

I heard SB had some DNS issues and want to know if it's just me or is someone else also facing this problem.

I think you answered your own question here--problems on the site will definitely cause problems for FFF.

That said, I didn't see the problems in FFF for myself. But that's because I was out and about yesterday instead of home downloading fics.

[JimmXinu]

New Version Released

Version 3.9.0 - 25 Jun 2019
- Fix for regression on XF not-first index post.
- Fix for a corner-case global_cache written by py2, then read by py3 bug.
- Update included dependencies -- mostly for Calibre 2.85.1.
- adapter_asianfanficscom json fixes - thanks oh45454545
- adapter_asianfanficscom foreword json fix - thanks oh45454545
- Change metadata separator to ' & ' when filling 'Contains names' custom columns.
- Fix for adapter_asianfanficscom change for views metadata
- Fix for site change adapter_novelonlinefullcom
- More fixes for adapter_asianfanficscom site now giving different HTML to FFF.
- Fix for some(older?) adapter_asianfanficscom stories not have json links.
- Fixes for site changes adapter_asianfanficscom
- Fix for extended chars in text email imap fetchs on py3.
- Show debug output BeautifulSoup version.
- adapter_mediaminerorg: Fix for not finding a block in chapter download.
- Fix for newer BS4/soupsieve enforcing CSS selector rules about :/ chars.
- Fix for site date change adapter_wuxiaworldcom

[serana]

Originally Posted by JimmXinu

(multi-reply)
This actually depends on number of chapters, how the site assigns chapter urls, and (possibly) what the author did.

FFF's default update, Update EPUB if New Chapters decides whether to update or not purely by chapter count. If the site has more chapters, it goes into update, if not, it doesn't. It would be more accurately named 'if MORE chapters'.

If the site has more chapters (or you invoked Update EPUB Always), FFF looks at the chapter URLs fetched from the site and the chapter URLs saved in the epub, and fetches any chapters it doesn't already have. If FFF already has a chapter downloaded for a chapter URL, it does not download it again.

So if an author deletes a chapter and adds another one on a site like ffnet that simply numbers chapters, FFF has no way to know. On a site like AO3, the same operation will have a different chapter number and be detected and downloaded.

On sites like SV/SB, if an author removes a post from the threadmarks(aka chapter list) and adds a different one, it will be detected. But if they edit the existing post, it will not.

On large fics, I personally use Calibre's Edit book feature to remove chapters I know the author has changed before running FFF update again. I don't have a better solution at this time.

Sorry for the late reply and thank you very much for your answer! This has indeed explained a lot to me. I also found it quite interesting that different sites handle that differently! I'll have to keep it in mind depending on whether I download from FFN or AO3 .

Originally Posted by JimmXinu

(multi-reply)
If I understand you correctly, Update EPUB Always will do what you want.

However, if you created and set a Saved Metadata Column, you can use Update Calibre Metadata from Saved Metadata Column to fill in new custom columns using the metadata gathered on your last download.

Thank you! I'll have to read up on the Update Calibre Metadata from Saved Metadata Column and fiddle with it a bit, but if any questions remain, I'll come back to ask again .

I really appreciate all the time and work you dedicated to this plugin. I can't thank you and everyone else who's helping us all archiving and organising fanfiction!

[Ryune]

I recently grabbed a fic and found that my Kindle was choking at a certain point in the book. Does anyone know if the kindle has a major issue with zalgo text or if it's just the fic in question?

https://archiveofourown.org/works/17...pters/42105455

[JimmXinu]

Originally Posted by Ryune

I recently grabbed a fic and found that my Kindle was choking at a certain point in the book. Does anyone know if the kindle has a major issue with zalgo text or if it's just the fic in question?

https://archiveofourown.org/works/17...pters/42105455

From defaults.ini:

## Some authors use 'Zalgo' text--arbitrary and often excessive
## added/combined unicode markings--to indicate 'noise' of some kind.
## While a critical part of some languages, when over used it can also
## cause problems for some ebook readers, such as Kindle.
## https://stackoverflow.com/questions/6579844/how-does-zalgo-text-work
## This setting will reduce the number 'Zalgo' characters.  Set to 0
## to remove all of them, -1 or leave undefined for no limit.
## Note: This will also remove other 'marked' unicode characters. If
## set to 0, café will become cafe, déjà -> deja, etc.  1 will keep
## single marks and is the recommended setting if you use it.
#max_zalgo:1

So put max_zalgo:1 (or a few more if you prefer) in your personal.ini and re-download (Overwrite Always) that story. It's up to you if you want the setting in [defaults] (affect all stories), [archiveofourown.org] (all AO3 stories) or [https://archiveofourown.org/works/17843429/] (just this story.)

[Ryune]

Awesome, thanks.

[SMNonMR]

Just curious, do you think the Gmail API lockdown mentioned in this article will affect FFF's email access? I noticed a while ago that Gmail prompted me to turn off "less secure app access". When I did that, FFF could no longer read emails. I turned that setting off but now I'm concerned that it will happen again.

https://arstechnica.com/gadgets/2019...rting-july-15/

[JimmXinu]

I don't think so.

IMAP4 is a standard email access protocol that's used by most mail clients.

[SMNonMR]

That's a relief! Thanks!

[Arkeus]

Wuxiaworld.co is doing error messages. An example would be this:

'NoneType' object is not callable
https://www.wuxiaworld.co/It-s-Not-E...to-the-Future/

From what I can see, anything on the site does this.

[JimmXinu]

The very last chapter of that story returns an empty page. This looks like a site problem, not a problem with FanFicFare.

And I don't think it's effecting all stories, because I picked one at random and it worked fine.

You can either limit your download with a chapter range, ie download using URL:
https://www.wuxiaworld.co/It-s-Not-E...to-the-Future/[-574]

...Or use this setting to continue on in the face of chapter errors:

[www.wuxiaworld.co]
continue_on_chapter_error:true

If you do that, your ebook will contain a chapter there, but the chapter title will contain (CHAPTER ERROR) and the text will be a report of the error:

575. Volume 2 575 He Is Ling Lan!(CHAPTER ERROR)

Error

FanFicFare failed to download this chapter. Because continue_on_chapter_error is set to true, the download continued.

Chapter URL:
https://www.wuxiaworld.co/It-s-Not-E...e/2072007.html

Error:
Traceback (most recent call last):
File "C:\Users\user\Desktop\nook\FanFicFare\fanficfare\ adapters\base_adapter.py", line 230, in getStory
data = self.getChapterTextNum(url,index)
File "C:\Users\user\Desktop\nook\FanFicFare\fanficfare\ adapters\base_adapter.py", line 381, in getChapterTextNum
return self.getChapterText(url)
File "C:\Users\user\Desktop\nook\FanFicFare\fanficfare\ adapters\adapter_wuxiaworldco.py", line 131, in getChapterText
for script in content('script'):
TypeError: 'NoneType' object is not callable

You will need to manually remove that chapter with Calibre's Edit book or similar before FFF will try to download that chapter again on update.

Hmmm. Perhaps I will look into adding an option for FFF to ignore chapters with '(CHAPTER ERROR)' in the title on update...

[Arkeus]

huh, should have checked if there was a site error, thanks.