[GUI Plugin] FanFicFare

[kolbinski]

When you say update translations for the plugin what do you mean?

[db105]

I just installed this Calibre plugin, tried to download from URL, and I get this error message:

Status: Error
Title: Unknown
Author: Unknown
Comment: HTTP Error in FFF '403 Client Error: Forbidden for url: https://www.fanfiction.net/s/7780036/1/The-North-Remembers'(403)
URL: https://www.fanfiction.net/s/7780036/1/The-North-Remembers

So, is FanFicFare forbidden to download from www.fanfiction.net, or am I doing something wrong?

Thanks for any help!

[ownedbycats]

You're likely running into Cloudflare.

[JimmXinu]

Originally Posted by kolbinski

When you say update translations for the plugin what do you mean?

FanFicFare has translations for it's UI strings for other languages as provided by volunteers at transifex (Some Calibre plugins, FFF specific link--probably need to make an account to see).

I use a script to pull out all the UI strings, volunteers on transifex translate them into various languages, I use a different script to pull any language with a significant number of translated strings, the translations are included in the plugin.

[sandmanwake]

webnovel downloads is throwing errors.

Debug log:

Spoiler:

EpubMerge: DEBUG: 2022-04-22 18:45:21,685: calibre_plugins.epubmerge.epubmerge_plugin(155): macmenuhack file_path:C:\Users\username\AppData\Roaming\calibr e\plugins\fanficfare_macmenuhack.txt
FFF: DEBUG: 2022-04-22 18:45:21,695: calibre_plugins.fanficfare_plugin.fff_plugin(218): Plugin FanFicFare macmenuhack file_path:C:\Users\username\AppData\Roaming\calibr e\plugins\fanficfare_macmenuhack.txt
calibre Debug log
calibre 5.26 embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: EpubMerge (2, 13, 0) && EpubSplit (3, 3, 0) && FanFicFare (4, 11, 12)
calibre 5.26 embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: EpubMerge (2, 13, 0) && EpubSplit (3, 3, 0) && FanFicFare (4, 11, 12)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 94.14671814671814 x 94.07407407407408
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.52] splash screen shown
[0.52] Initializing db...
[1.38] db initialized
[1.38] Constructing main UI...
[3.30] main UI initialized...
[3.30] Hiding splash screen
Starting QuickView
FFF: DEBUG: 2022-04-22 18:45:34,665: calibre_plugins.fanficfare_plugin.dialogs(497): self.extraoptions['anthology_url']:NOT FOUND
FFF: DEBUG: 2022-04-22 18:45:34,667: calibre_plugins.fanficfare_plugin.fff_plugin(1129) : FanFicFare v4.11.12
FFF: INFO: 2022-04-22 18:45:34,684: calibre_plugins.fanficfare_plugin.prefs(214): Using default settings
FFF: DEBUG: 2022-04-22 18:45:34,699: configurable.py(1035): use_browser_cache:
FFF: DEBUG: 2022-04-22 18:45:34,699: configurable.py(1049): use_basic_cache:true
FFF: DEBUG: 2022-04-22 18:45:34,701: adapter_webnovelcom.py(85): https://www.webnovel.com/book/21542016806826305
FFF: DEBUG: 2022-04-22 18:45:34,701: fetcher.py(234):
========== MISS (GET) BasicCache
https://www.webnovel.com/book/21542016806826305
FFF: DEBUG: 2022-04-22 18:45:34,702: fetcher.py(469):
---------- REQ (GET) RequestsFetcher
https://www.webnovel.com/book/21542016806826305
FFF: DEBUG: 2022-04-22 18:45:35,168: fetcher.py(482): response code:200
FFF: DEBUG: 2022-04-22 18:45:35,169: requestable.py(50): Encoding:utf8
FFF: ERROR: 2022-04-22 18:45:35,331: calibre_plugins.fanficfare_plugin.dialogs(714): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': "'NoneType' object has no attribute 'get_text'", 'url': 'https://www.webnovel.com/book/21542016806826305', 'site': 'www.webnovel.com', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://www.webnovel.com/book/21542016806826305[None-None]', 'listorder': 0, 'collision': 'Update EPUB if New Chapters'}:'NoneType' object has no attribute 'get_text'
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.dialogs", line 701, in do_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1343, in prep_download_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1215, in get_story_metadata_only
File "C:\Users\username\AppData\Roaming\calibre\plugins \FanFicFare.zip\fanficfare\adapters\base_adapter.p y", line 308, in getStoryMetadataOnly
File "C:\Users\username\AppData\Roaming\calibre\plugins \FanFicFare.zip\fanficfare\adapters\adapter_webnov elcom.py", line 109, in doExtractChapterUrlsAndMetadata
File "C:\Users\username\AppData\Roaming\calibre\plugins \FanFicFare.zip\fanficfare\htmlcleanup.py", line 71, in stripHTML
AttributeError: 'NoneType' object has no attribute 'get_text'
[13.38] splash screen hidden
[13.38] Started up in 13.38 seconds with 13813 books

[nephtys59]

Don't know if this question has already been asked, but can't find it through the search engine. Is there a way to prevent the plugin from compressing the images too much?
There's a fic on AO3 including some images from a chat and they came out all blurred and unreadable, while -if I download the epub directly from AO3- they're perfectly fine.
In fact, the epub size created by FanFicFare is 1 MB while the original EPUB is 5,6 MB.
I link 2 images for comparison:

FanficFare (unreadable): link

Original: link

Thanks in advance for helping me

[JimmXinu]

Originally Posted by sandmanwake

webnovel downloads is throwing errors.

Looks like the site has changed some more. I'll look at it, but it may be next week before I have time.

Originally Posted by nephtys59

Don't know if this question has already been asked, but can't find it through the search engine. Is there a way to prevent the plugin from compressing the images too much?
There's a fic on AO3 including some images from a chat and they came out all blurred and unreadable, while -if I download the epub directly from AO3- they're perfectly fine.
In fact, the epub size created by FanFicFare is 1 MB while the original EPUB is 5,6 MB.
I link 2 images for comparison:

FanficFare (unreadable): link

Original: link

Thanks in advance for helping me

I edited your post, MR doesn't like large images in-line.

FFF, by default, resizes images to be within the image_max_size setting (default 580, 725), converts to jpg, and removes transparency for e-ink e-readers. You can adjust those settings or set no_image_processing: true to use images as-is.

Note that if you convert the epub to another format, like azw3 for Kindle, Calibre's convert also resizes images based on the Convert -> Page set -> Output profile. Use 'Tablet' for no image resizing.

[tricklem]

Just switching my default browser to Microsoft Edge and can't find where the cache is stored so that I can update FFF. Anyone have any suggestions on where I could find this? Google hasn't been any help with this.

[ownedbycats]

Originally Posted by tricklem

Just switching my default browser to Microsoft Edge and can't find where the cache is stored so that I can update FFF. Anyone have any suggestions on where I could find this? Google hasn't been any help with this.

Put %LocalAppData% in your start menu and press enter, then look for the \Packages\ subfolder. Then look for Microsoft.MicrosoftEdge_xxxx (the last bit will be random).

According to this page it will be in \AC\#!001\MicrosoftEdge\Cache\

[JimmXinu]

New Test Version Posted

2022-04-25
- adapter_webnovelcom: Fixes for site changes. #828
- adapter_fictionhuntcom: Fixes for site changes.

[ILB]

I get the following error on story https://www.fanfiction.net/s/14072096/1/

Spoiler:

calibre, version 5.41.0
ERROR: Unhandled exception: <b>TypeError</b>:'&lt;' not supported between instances of 'int' and 'NoneType'

calibre 5.41 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Annotations (1, 17, 0) && Copy Cover To Device (2, 0, 4) && Count Pages (1, 11, 2) && EpubMerge (2, 13, 0) && FanFicFare (4, 11, 4) && Favourites Menu (1, 1, 0) && Find Duplicates (1, 9, 4) && Generate Cover (2, 1, 0) && Kindle Collections (1, 7, 29) && Manage Series (1, 3, 0) && Quick Preferences (1, 5, 0) && Reading List (1, 8, 0) && Similar Stories (1, 0, 58) && SmartEject (2, 4, 0) && User Category (1, 5, 5)
Traceback (most recent call last):
File "calibre\gui2\proceed.py", line 208, in do_ask_question
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 2146, in update_error_column
File "calibre_plugins.fanficfare_plugin.dialogs", line 639, in LoopProgressDialog
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1887, in update_books_finish
File "calibre\db\view.py", line 393, in set_marked_ids
File "calibre\gui2\__init__.py", line 565, in __call__
File "calibre\gui2\library\views.py", line 950, in marked_changed
File "calibre\gui2\library\models.py", line 400, in refresh_rows
File "calibre\gui2\library\models.py", line 74, in group_numbers
TypeError: '<' not supported between instances of 'int' and 'NoneType'

[JimmXinu]

Originally Posted by ILB

I get the following error on story https://www.fanfiction.net/s/14072096/1/

Spoiler:

calibre, version 5.41.0
ERROR: Unhandled exception: <b>TypeError</b>:'&lt;' not supported between instances of 'int' and 'NoneType'

calibre 5.41 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Annotations (1, 17, 0) && Copy Cover To Device (2, 0, 4) && Count Pages (1, 11, 2) && EpubMerge (2, 13, 0) && FanFicFare (4, 11, 4) && Favourites Menu (1, 1, 0) && Find Duplicates (1, 9, 4) && Generate Cover (2, 1, 0) && Kindle Collections (1, 7, 29) && Manage Series (1, 3, 0) && Quick Preferences (1, 5, 0) && Reading List (1, 8, 0) && Similar Stories (1, 0, 58) && SmartEject (2, 4, 0) && User Category (1, 5, 5)
Traceback (most recent call last):
File "calibre\gui2\proceed.py", line 208, in do_ask_question
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 2146, in update_error_column
File "calibre_plugins.fanficfare_plugin.dialogs", line 639, in LoopProgressDialog
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1887, in update_books_finish
File "calibre\db\view.py", line 393, in set_marked_ids
File "calibre\gui2\__init__.py", line 565, in __call__
File "calibre\gui2\library\views.py", line 950, in marked_changed
File "calibre\gui2\library\models.py", line 400, in refresh_rows
File "calibre\gui2\library\models.py", line 74, in group_numbers
TypeError: '<' not supported between instances of 'int' and 'NoneType'

I'm not able to duplicate.

That's the code that calls into calibre to marked updated books with db.set_marked_ids(marked_ids).

Considering the Calibre marking code just got a significant update, I suspect it's either a problem with that, or with how FFF calls it.

How many downloads/updates were you doing at the time?

[ownedbycats]

Question: is it possible to do a replace_metadata based on an author URL (rather than the author name)?

edit: actually, that might not work, as AO3 seems to use the author name in the URL rather than some sort of uid. Since it's one story I guess story URL would work.

[chaley]

Originally Posted by JimmXinu

I'm not able to duplicate.

That's the code that calls into calibre to marked updated books with db.set_marked_ids(marked_ids).

Considering the Calibre marking code just got a significant update, I suspect it's either a problem with that, or with how FFF calls it.

How many downloads/updates were you doing at the time?

Looking at the code and the trace it appears that FFF is calling set_marked_ids() with a dict that contains None values, as in {1234:None}. It is likely that this was converted to {1234:True} before the changes, while now for a key:value pair the value is preserved. The dict passed to set_marked_ids() should contain only books that are to be marked using a text string or the value "true" (not a boolean True) if a general (non-text) mark is desired. Books that are to be unmarked should be left out of the dict.

[JimmXinu]

Originally Posted by chaley

Looking at the code and the trace it appears that FFF is calling set_marked_ids() with a dict that contains None values, as in {1234:None}. It is likely that this was converted to {1234:True} before the changes, while now for a key:value pair the value is preserved. The dict passed to set_marked_ids() should contain only books that are to be marked using a text string or the value "true" (not a boolean True) if a general (non-text) mark is desired. Books that are to be unmarked should be left out of the dict.

That was my thought as well from the initial error "TypeError: '<' not supported between instances of 'int' and 'NoneType'".

But the FFF code only calls set_marked_ids() with a dict that contains id->string value, or an empty dict() to clear all marks. No where does it set a boolean or a None into the dict. And I haven't been able to reproduce the reported issue.

ILB - What are your FFF 'Mark' settings on the FFF General config tab, and how did you produce the error? Were you doing anything else that used Marks at the time?


This new named marking UI conflicts a bit with the way Reading List PI and FFF have used a unique, numbered named mark on each book to both mark and order. I wonder if having hundreds or thousands of uniquely named marks (from viewing a Reading List) will cause problems? My 5k+ library fills the screen on 'Show marked books with text' but otherwise is fine.

FYI, the new 'Mark with text label' action will allow an all whitespace label. And treats one space and two spaces as different labels. Doesn't seem to hurt anything, but may not be desired.