[GUI Plugin] FanFicFare

[ownedbycats]

Originally Posted by JimmXinu

More info: There are three times in FFF plugin that self.gui.proceed_question() will be called:

1. Successful update or new download of normal book(s), and (optionally) update of error column for any failures.
2. Successful update or new download of Anthology book. (Update/download aborts if no stories in the anthology succeeded.)
3. Update of error column only when all updates/downloads fail (optionally).

If all of these cases emit the same signal, is that useful? Or is someone going to ask to have them separated immediately?

I think some separation for new and updated books makes sense.

[JimmXinu]

Originally Posted by bigwoof

I have a chain that I run only when new books are added (it sets all the tags properly). so something that runs when new books are added would be great.

Originally Posted by ownedbycats

I think some separation for new and updated books makes sense.

That's not actually on offer. FFF can and does download new stories and update existing stories at the same time. And errors, all in the same library update.

I assume you'll have to design your Action Chains to find and select the books to operate on based on the metadata written to the library columns by FFF.

I honestly haven't pursued a deep understanding of Action Chains.

FFF has options to mark Success/Fail/Chapter Error books and then 'show' them, but even then, the books are not selected.

[ownedbycats]

In my specific case, I design my chains to be as non-destructive as possible (I've accidentally run it over non-fanfics and gotten zero changes), but it might be a concern for other users.

[JimmXinu]

New Test Version

2024-05-28
- Add download_finished_signal for Action Chains #1073
- New Site: touchfluffytail.org (#1071) thanks dbhmw

The FFF plugin has a signal it will emit after the "proceed with updating your library" dialog.

The code for the Action Chains Module I tested with is:

from calibre_plugins.action_chains.events import ChainEvent

class SearchChanged(ChainEvent):

    # replace with the name of your event
    name = 'FFF done'

    def get_event_signal(self):
        return self.gui.iactions['FanFicFare'].download_finished_signal

All I've done with it is call a chain that printed 'HERE!'.

[johansolo]

First of all, this plugin is wonderfull!!!

And now with my question.

I have downloaded several stories from Scribblehub. (The conversion was perfect I only had to edit personal.ini to modify the css and add indentation.)

But I use KOReader to read epubs and the cover is tiny.

I have seen that cover.xhtml does not have "<link href="stylesheet.css" type="text/css" rel="stylesheet"/>" and "<img src="images/cover.jpg" alt="cover "/>" has no class.

Is there a way to add that code automatically so later I can tinkering with personal.ini to adjust css?

Thanks.

SFW link
https://www.scribblehub.com/series/1...iverse-litrpg/

Screenshots from KOReader and Calibre reader attached. The KOReader screenshot my appear large but the e-reader actually has only 6''.

EDIT: Sorry for my English, is not my native language.

[JimmXinu]

Originally Posted by johansolo

Is there a way to add that code automatically so later I can tinkering with personal.ini to adjust css?

You can customize the HTML generated for cover.xhtml using the cover_content setting. Changing it to include <link href="stylesheet.css" type="text/css" rel="stylesheet"/> should be easy enough.

See CustomOutput wiki page.

Also, be aware that by default FFF reduces the size of large cover images to a 'reasonable' default:

[epub]
# ...
## Resize images down to width, height, preserving aspect ratio.
## Nook size, with margin.
image_max_size: 580, 725

Your example already has a smaller image, so it wouldn't matter in that case.

[ownedbycats]

Originally Posted by JimmXinu

The FFF plugin has a signal it will emit after the "proceed with updating your library" dialog.

The code for the Action Chains Module I tested with is:

from calibre_plugins.action_chains.events import ChainEvent

class SearchChanged(ChainEvent):

    # replace with the name of your event
    name = 'FFF done'

    def get_event_signal(self):
        return self.gui.iactions['FanFicFare'].download_finished_signal

All I've done with it is call a chain that printed 'HERE!'.

If 'mark added/updated books when finished' is enabled, these selection modifier settings will select all updated books.



I created a chain with this selection modifier and then a chain caller to my fanfic cleanup chain. Seems to work as intended.

[JimmXinu]

That reminds me about something that could be helpful.

FFF actually uses different marks for successful, failed and chapter error, as well as for an updated/created anthology:

• fff_success
• fff_failed
• fff_chapter_error
• fff_anthology

Technically, each book is marked with a sequence number like, "fff_success_0001" so that order is also preserved.

[johansolo]

Originally Posted by JimmXinu

You can customize the HTML generated for cover.xhtml using the cover_content setting. Changing it to include <link href="stylesheet.css" type="text/css" rel="stylesheet"/> should be easy enough.

See CustomOutput wiki page.

I didn't know about the wiki. So many posibilities now...

About the cover thing. I only had to change the img line.

cover_content:
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
 <head>
 <title>Cover</title>
 <style type="text/css" title="override_css">
 @page {padding: 0pt; margin:0pt}
 body { text-align: center; padding:0pt; margin: 0pt; }
 div { margin: 0pt; padding: 0pt; }
 </style>
 </head>
 <body class="fff_coverpage">
 <div>
 <svg xmlns="http://www.w3.org/2000/svg" height="100%%" preserveAspectRatio="xMidYMid meet" version="1.1" viewBox="0 0 1200 1800" width="100%%" xmlns:xlink="http://www.w3.org/1999/xlink"><image height="1800" width="1200" xlink:href="${coverimg}"/></svg>
 </div>
 </body>
 </html>

And now is perfect for KOReader.

Thanks.

[penumbria]

Since Wednesday night I have been having a problem updating stories through FFF coming from AO3. It was fine Wednesday afternoon but Wednesday night and all day Thursday, I have been getting 403 errors.
This is the error text from the clipboard and it is happening with multiple different stories.

Status
Title
Author
Comment
URL
Error
Flight of Sanctuary
DianneRose2016
HTTP Error in FFF '403 Client Error: Forbidden for url: https://archiveofourown.org/works/55...=true'(403)
https://archiveofourown.org/works/55180027


I didn't change anything between Wednesday afternoon and evening so I don't know what happened. Is this an issue on my end somehow or is it AO3? Is there a fix-it or workaround?
I'm not hugely tech-savvy but I can follow clear directions when needed.

[JimmXinu]

The only reasons I recall seeing that from AO3 are when the site is having problems, or the user did 'too many' downloads 'too quickly'. Unfortunately, we don't know the numbers for 'too many' or 'too quickly'.

How many downloads did you do before seeing that?

[penumbria]

It started when I hadn't done any for hours. And it is still happening even now after hours of break time in between attempts.

[JimmXinu]

We don't know what constitutes 'too many', 'too quickly', or how long it takes to cool down--I would assume day(s), not just hours.

It's also possible it's someone else 'nearby' in your network. Some people have had success rebooting routers to get new outside facing IP addresses or using VPNs.

I don't recall if FFF's Browser Cache feature works with AO3 or not.

And worst case, you can always download an epub or other ebook format from AO3, unlike a lot of other sites.

[ownedbycats]

Depending on your ISP, it's posisble that you got an IP address previously used by a bad actor. Also depending on your ISP, unplugging your modem for a bit and then reconnecting might get you a new one and help.

Some users did have some success with the cache, but that was when the site was being DDoS'd.

[vikarti]

How to fix issue with ficbook.net?
regular connection via CLI:
C:\Users\Vikarti Anatra>calibre-debug.exe --run-plugin FanFicFare https://ficbook.net/readfic/018aa4e6...9-8d2beb0ad5d4
Traceback (most recent call last):
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\fetcher_requests.py", line 128, in request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\requests\models.py", line 943, in raise_for_status
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://ficbook.net/readfic/018aa4e6...9-8d2beb0ad5d4

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\debug.py", line 289, in main
File "calibre_plugins.fanficfare_plugin.__init__", line 146, in cli_main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 346, in main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 322, in dispatch
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 437, in do_download
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 325, in getStoryMetadataOnly
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 429, in doExtractChapterUrlsAndMetadata
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\adapter_ficbooknet.py", line 80, in extractChapterUrlsAndMetadata
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\requestable.py", line 119, in get_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\requestable.py", line 111, in get_request_redirected
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\base_fetcher.py", line 133, in get_request_redirected
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\cache_basic.py", line 122, in fetcher_do_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\decorators.py", line 102, in fetcher_do_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\base_fetcher.py", line 106, in do_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\fetcher_requests.py", line 149, in request
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF '403 Client Error: Forbidden for url: https://ficbook.net/readfic/018aa4e6...ad5d4'(403)

attempt to use flareslolver with modifications in personal.ini

use_flaresolverr_proxy:true
#run via docker desktop using docker run -d --name=flaresolverr -p 8191:8191 -e LOG_LEVEL=info flaresolverr/flaresolverr:latest
results in flaresolverr seeing icoming request, saying version_main cannot be converted to an integer and "challenge not detected"

results in
C:\Users\Vikarti Anatra>calibre-debug.exe --run-plugin FanFicFare https://ficbook.net/readfic/018aa4e6...9-8d2beb0ad5d4
FFF: WARNING: 2024-06-01 12:37:42,287: configurable.py(1059): FlareSolverr v2+ doesn't work with images: include_images automatically set false
FFF: WARNING: 2024-06-01 12:37:42,287: configurable.py(1060): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images
FFF: WARNING: 2024-06-01 12:37:42,287: configurable.py(1061): Set use_flaresolverr_proxy:directimages to download images directly while using FlareSolver
Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\debug.py", line 289, in main
File "calibre_plugins.fanficfare_plugin.__init__", line 146, in cli_main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 346, in main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 322, in dispatch
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 437, in do_download
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 325, in getStoryMetadataOnly
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 429, in doExtractChapterUrlsAndMetadata
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\adapter_ficbooknet.py", line 237, in extractChapterUrlsAndMetadata
TypeError: 'NoneType' object is not subscriptable

browser doesn't see any challenges on ficbook.net

usign vpn doesn't help (same set of errors as before)
any advice would be anticipated.

starting flaresolver as docker run -d --name=flaresolverr -p 8191:8191 -e LOG_LEVEL=debug -e LOG_HTML=true flaresolverr/flaresolverr:latest
and connecting to containter for logs shows ficbook's html IS being here


using flaresloverr with fanfiction.net via fanficfare detects challenge correctly