[GUI Plugin] FanFicFare

[ackomb]

Getting this error after downloading, and I can't figure what the trigger is because the behavior afterwards keeps changing,

Spoiler:

calibre, version 5.42.0
ERROR: Unhandled exception: <b>TypeError</b>:'&lt;' not supported between instances of 'int' and 'NoneType'

calibre 5.42 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.22000 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22000')
Python 3.8.5
Windows: ('10', '10.0.22000', '', 'Multiprocessor Free')
Interface language: en_GB
Successfully initialized third party plugins: Action Chains (1, 16, 3) && BookFusion Plugin (0, 6, 3) && Count Pages (1, 11, 2) && FanFicFare (4, 12, 4) && Find Duplicates (1, 9, 5) && Generate Cover (2, 1, 0) && Job Spy (1, 0, 190) && Mass Search-Replace (1, 4, 2) && Modify ePub (1, 7, 3) && Reading List (1, 9, 0) && User Category (1, 5, 5) && View Manager (1, 8, 0)
Traceback (most recent call last):
File "calibre\gui2\proceed.py", line 208, in do_ask_question
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 2126, in do_download_list_update
File "calibre_plugins.fanficfare_plugin.dialogs", line 639, in LoopProgressDialog
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1887, in update_books_finish
File "calibre\db\view.py", line 393, in set_marked_ids
File "calibre\gui2\__init__.py", line 585, in __call__
File "calibre\gui2\library\views.py", line 968, in marked_changed
File "calibre\gui2\library\models.py", line 400, in refresh_rows
File "calibre\gui2\library\models.py", line 74, in group_numbers
TypeError: '<' not supported between instances of 'int' and 'NoneType'

For example first it downloads, I'm not sure what to call it, but the shell without any formats?? The error is as follows:

Spoiler:

calibre, version 5.42.0
ERROR: Cannot view: Snow Scraper has no available formats.

Then I try again, same message as the first but now it contains the regular metadata and epub.
Doesn't matter which story it happens to all the stories accumulated into one action.

[JimmXinu]

Originally Posted by ackomb

Getting this error after downloading, and I can't figure what the trigger is because the behavior afterwards keeps changing,

Spoiler:

calibre, version 5.42.0
ERROR: Unhandled exception: <b>TypeError</b>:'&lt;' not supported between instances of 'int' and 'NoneType'

calibre 5.42 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.22000 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22000')
Python 3.8.5
Windows: ('10', '10.0.22000', '', 'Multiprocessor Free')
Interface language: en_GB
Successfully initialized third party plugins: Action Chains (1, 16, 3) && BookFusion Plugin (0, 6, 3) && Count Pages (1, 11, 2) && FanFicFare (4, 12, 4) && Find Duplicates (1, 9, 5) && Generate Cover (2, 1, 0) && Job Spy (1, 0, 190) && Mass Search-Replace (1, 4, 2) && Modify ePub (1, 7, 3) && Reading List (1, 9, 0) && User Category (1, 5, 5) && View Manager (1, 8, 0)
Traceback (most recent call last):
File "calibre\gui2\proceed.py", line 208, in do_ask_question
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 2126, in do_download_list_update
File "calibre_plugins.fanficfare_plugin.dialogs", line 639, in LoopProgressDialog
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1887, in update_books_finish
File "calibre\db\view.py", line 393, in set_marked_ids
File "calibre\gui2\__init__.py", line 585, in __call__
File "calibre\gui2\library\views.py", line 968, in marked_changed
File "calibre\gui2\library\models.py", line 400, in refresh_rows
File "calibre\gui2\library\models.py", line 74, in group_numbers
TypeError: '<' not supported between instances of 'int' and 'NoneType'

That's an issue with the new Marking features in Calibre. From #833:

This was previously reported in mobileread. It was determined to be a corner case issue with recent changes to Calibre 'Mark' feature.

I haven't heard since anything since. To the best of my knowledge, it's not something I can fix in FFF and I can't reproduce it.

You may need to turn off the 'Mark' and 'Show Marked' features in FFF in the meantime.

I'll leave this issue open for the time being.

Originally Posted by ackomb

For example first it downloads, I'm not sure what to call it, but the shell without any formats?? The error is as follows:

Spoiler:

calibre, version 5.42.0
ERROR: Cannot view: Snow Scraper has no available formats.

Then I try again, same message as the first but now it contains the regular metadata and epub.
Doesn't matter which story it happens to all the stories accumulated into one action.

I have no idea what that is--never heard of 'Snow Scraper' before. Sounds more like a Calibre issue than an FFF issue. Possibly something in an unexpected state due to the problem above??

[ackomb]

Originally Posted by JimmXinu

That's an issue with the new Marking features in Calibre.

Okay, then that message appears to be unrelated to the issue I had. Problem is I can't replicate it on purpose to run it in a debug log.

Maybe a description rings a bell.

I attempted to download 4 stories, all four were fine according to the log, however when I pressed "yes" to add them into the library they were empty shells. It showed the title and the author, but the was no EPUB file attached to it.

--

Originally Posted by JimmXinu

I have no idea what that is--never heard of 'Snow Scraper' before. Sounds more like a Calibre issue than an FFF issue. Possibly something in an unexpected state due to the problem above??

Unfortunate choice of story in this event.
The story name is "Snow Scraper" I assume the error reverts to the EPUB not being available.
I just didn't know how to describe it properly so I included it. Sorry if it caused any confusion.

[JimmXinu]

Full disk? RAM going bad? If it isn't generating log errors, I've no idea, unfortunately.

Personally, I always run Calibre in debug mode, even when I'm not working on code, just to catch anything that may happen.

[ackomb]

Unrelated to the above (I think), when looking at the job log during download I noticed a strange thing at the top

Spoiler:

Failed to initialize plugin: 'C:\\Users\\****\\AppData\\Roaming\\calibre\\plugi ns\\Fantastic Fiction.zip'

But this is not a plugin I installed, nor can I find it in the plugin list.

[Ser4nb2LUY6e]

The novelfull adapter also works for allnovelfull.com when copied and a search and replace novelfull -> allnovelfull applied. (There are stories on allnovelfull that aren't on novelfull.)

As with novelfull.com, it's necessary to use flaresolverr.

[JimmXinu]

Fantastic Fiction is a known plugin.

I would speculate that you did once download it and it failed, or is out of date.

If it's only showing up in the jobs output, I would further speculate that maybe you have a confused/damaged Calibre install that is some how partially using code or executables for another, older Calibre install.

I vaguely recall that happening before, and I know that I've had Windows become confused start using book reader from older, portable versions when double clicking say .epub files.

Repair or uninstall/re-install of Calibre may help. YMMV, this is outside my area of knowledge.

[JimmXinu]

New Test Version Posted

2022-05-22
- adapter_quotevcom: Update collection of searchtags

[JimmXinu]

Originally Posted by Ser4nb2LUY6e

The novelfull adapter also works for allnovelfull.com when copied and a search and replace novelfull -> allnovelfull applied. (There are stories on allnovelfull that aren't on novelfull.)

As with novelfull.com, it's necessary to use flaresolverr.

While I continue to support existing adapters that now require flaresolverr, I don't want to add new ones.

Search the thread for readnovelfull to see the full discussion of the precedent:

Originally Posted by JimmXinu

- Contributed readnovelfull.com code is not included. While I would like to encourage code contributions, I don't want to add a new site that requires a proxy right from the start.

[Ser4nb2LUY6e]

Fair enough. I'll just keep a private fork of FanFicFare for myself from now on then.

Originally Posted by JimmXinu

While I continue to support existing adapters that now require flaresolverr, I don't want to add new ones.

Search the thread for readnovelfull to see the full discussion of the precedent:

[Ser4nb2LUY6e]

Actually, technically, I think the site doesn't require flaresolverr, because that always says "Cloudflare not detected".

When used without flaresolverr, the download fails with a "403: Client not permitted" error or something like that, so it might be that the site is just filtering access based on user agent or something like that.

Originally Posted by JimmXinu

While I continue to support existing adapters that now require flaresolverr, I don't want to add new ones.

Search the thread for readnovelfull to see the full discussion of the precedent:

[chaley]

Originally Posted by JimmXinu

That's an issue with the new Marking features in Calibre. From #833:

Sorry -- I forgot about this.

I *think* it can happen only if

1. There are no marked books.
2. A set of book IDs are marked, where at least one of the IDs is for a deleted or non-existent book.

This sequence avoids the check for deleted books (book_id is None).

Note that if there are other marks then the check is done. The list of marked books must be empty before the problematic book is added.

There are some other errors in my changed code that could cause problems. The biggest is one place where I confused the book_id with the booklist row number. Sigh.

EDIT for explanation: many years ago, V0 .. V2 I think, calibre used the row number in the GUI for the book id, changing the 'real' id into the row number when needed. This changed with Kovid's new database code in V3 (I think), where the real book id became the one-true-ID. However, there is still a pile of legacy code in the GUI that uses the 'old' system. I confused myself when I looked at the API and chose the wrong method. As I said, sigh...

[ownedbycats]

Originally Posted by Ser4nb2LUY6e

Actually, technically, I think the site doesn't require flaresolverr, because that always says "Cloudflare not detected".

ffnet does too, even though it clearly is behind Cloudflare.

[JimmXinu]

New Test Version Posted

2022-05-22
- Don't set marked in Calibre when book_id is None #833
- adapter_quotevcom: Update collection of searchtags

Originally Posted by Ser4nb2LUY6e

Actually, technically, I think the site doesn't require flaresolverr, because that always says "Cloudflare not detected".

When used without flaresolverr, the download fails with a "403: Client not permitted" error or something like that, so it might be that the site is just filtering access based on user agent or something like that.

I think that "Cloudflare not detected" shows up when CF isn't showing up visibly. IE, first request, CF shows up, does JS nonsense or captcha, then doesn't show up for the next X requests or Y time.

You could also try use_cloudscraper:true, but that doesn't work for anything but the lowest levels of CF on 1-2 sites last I looked.

Originally Posted by chaley

Sorry -- I forgot about this.

I *think* it can happen only if

1. There are no marked books.
2. A set of book IDs are marked, where at least one of the IDs is for a deleted or non-existent book.

...

I was going to repeat that I didn't think FFF could set 'None' for book_id... but then I found that it could, in the relatively case of a failed new download with 'mark failures' checked. Which I would have sworn I had previously tested.

So, my apologies to chaley and all for not figuring out earlier that FFF can in fact, fix this issue. I was certain that I'd tested that.

[chaley]

Originally Posted by JimmXinu

I was going to repeat that I didn't think FFF could set 'None' for book_id... but then I found that it could, in the relatively case of a failed new download with 'mark failures' checked. Which I would have sworn I had previously tested.

So, my apologies to chaley and all for not figuring out earlier that FFF can in fact, fix this issue. I was certain that I'd tested that.

Not to worry. I have submitted changes to Kovid that will filter None values from marked_ids before attempting to use them. That will ensure that other paths, such as parallel jobs, don't get us back into the same situation.