[GUI Plugin] FanFicFare

[Kammal]

Hello, the plugin is not fetching the images from Questionable Questing that are under spoilers.

Here is a example from QQ(SFW):

<div class="ToggleTriggerAnchor bbCodeSpoilerContainer">
<button type="button" class="button bbCodeSpoilerButton ToggleTrigger Tooltip JsOnly"
title="Click to reveal or hide spoiler"
data-target="> .SpoilerTarget"><span>Spoiler: <span class="SpoilerTitle">Carol</span></span></button>
<div class="SpoilerTarget bbCodeSpoilerText"><img data-src="proxy.php?image=https%3A%2F%2Fi.pinimg.com%2F 474x%2F80%2F67%2F5b%2F80675b92b99d2b66ec1338054024 6490--fantasy-characters.jpg&amp;hash=fd0194ae2c0c5bf7b8ae4a1f96 bec08a" class="bbCodeImage LbImage lazyload" alt="[&#x200B;IMG]" data-url="https://i.pinimg.com/474x/80/67/5b/80675b92b99d2b66ec13380540246490--fantasy-characters.jpg" /><noscript><img src="proxy.php?image=https%3A%2F%2Fi.pinimg.com%2F 474x%2F80%2F67%2F5b%2F80675b92b99d2b66ec1338054024 6490--fantasy-characters.jpg&amp;hash=fd0194ae2c0c5bf7b8ae4a1f96 bec08a" alt="[&#x200B;IMG]"></noscript></div>
</div> <br />

and here is the output in the epub:

<div class="ToggleTriggerAnchor bbCodeSpoilerContainer">
<button class="button bbCodeSpoilerButton ToggleTrigger Tooltip JsOnly"><span>Spoiler: <span class="SpoilerTitle">Carol</span></span></button>
<div class="SpoilerTarget bbCodeSpoilerText"><img alt="[​IMG]" class="bbCodeImage LbImage lazyload"/></div>
</div>

[JimmXinu]

Originally Posted by Kammal

Hello, the plugin is not fetching the images from Questionable Questing that are under spoilers.

Here is a example from QQ(SFW):


and here is the output in the epub:

It's best to use [code], not [quote] tags for code blocks. And then wrap the [code] tags in [SPOILER] tags if they are large. Just FYI.

Can you PM me the story URL?

[JimmXinu]

New Test Version Posted

2021-08-20
- Fix for lazyload images in base_xenforoforum_adapter

[Ser4nb2LUY6e]

The adapter for wuxiaworld.site has stopped working.

It fails for all stories with:

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.jobs", line 260, in do_download_for_worker
  File "C:\Users\x\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 308, in getStoryMetadataOnly
  File "C:\Users\x\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 412, in doExtractChapterUrlsAndMetadata
  File "C:\Users\x\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\adapter_wuxiaworldsite.py", line 110, in extractChapterUrlsAndMetadata
AttributeError: 'NoneType' object has no attribute 'i'

[JimmXinu]

Originally Posted by Ser4nb2LUY6e

The adapter for wuxiaworld.site has stopped working.

I've opened issue #729.

If Kolbo5 (the original dev for that adapter) isn't interested or able to deal with it, I'll look into it more.

[Ser4nb2LUY6e]

Originally Posted by JimmXinu

I've opened issue #729.

If Kolbo5 (the original dev for that adapter) isn't interested or able to deal with it, I'll look into it more.

How is it generally handled when html is loaded by scripts after loading the actual page?

I looked at the adapter code and the site, and it seems to me that on the site now, the list of chapters is being loaded on demand by a script via a post request to https://wuxiaworld.site/wp-admin/admin-ajax.php

But the adapter code is simply calling:

data = self.get_request(self.url)
soup = self.make_soup(data)

and assumes after that the the html which is dynamically loaded by the script is present. Is get_request purely a simple http request, or does it provide a full environment to execute javascript?

[JimmXinu]

Originally Posted by Ser4nb2LUY6e

How is it generally handled when html is loaded by scripts after loading the actual page?
...

To quote the issue linked in my post:

The site wuxiaworld.site has changed significantly and breaks FFF's current code for parsing it.

The site is no longer serving the chapter list info in the HTML page and instead making a POST request in Javascript to .../admin-ajax.php to get the chapter list info.

The good news is that it appears the POST returns the chapter list as HTML that looks very similar to the previous chapter list code.

@Kolbo5, since you added this adapter to begin with, do you have the time (and interest) to deal with this?

You can search other adapters for post_request to see examples of how FFF performs POST requests.

FFF does not execute javascript.

[Book_fury]

How do you login to a website that uses Facebook signing on fanficfare

[Book_fury]

Originally Posted by Book_fury

How do you login to a website that uses Facebook signing on fanficfare

The website I want to log into is webnovel but it uses Facebook login and it doesn’t ask for me to is there a way around that

[JimmXinu]

FFF doesn't do login for webnovel.com. Nor is it likely to be added. I'm very leery of downloading user/pass protected material from original/translated content sites.

[Book_fury]

Originally Posted by JimmXinu

FFF doesn't do login for webnovel.com. Nor is it likely to be added. I'm very leery of downloading user/pass protected material from original/translated content sites.

I get that thanks anyway

[FaceDeer]

Long time since I've had anything go wrong with this plugin, so long time since my last comment. But alas, looks like the streak ended. I just noticed that none of the fanfics I've downloaded from Fimfiction since around July 1st or so have cover images. A few test downloads suggested that FFF is failing to fetch valid cover images, so I tried debugging a bit by scattering prints() about to see where it's going wrong. As far as I can see the adapter itself is working correctly - it finds the right URL and passes it to setCoverImage, getting back a return value that looks correct. Following execution down into addImgUrl in story.py shows it getting into the "# explicit cover, make the first image." block. At this point I'm well into code that I haven't poked around in before and I won't have much free time this evening to learn it, so I figured posting here was a good next step. I've randomly selected https://www.fimfiction.net/story/501587/ as a test example (it's recent and small) but the problem seems to apply to any story with a cover image. Given where I manage to get to in the code I don't think I've missed any obscure configuration settings anywhere, and this was working before with my existing config anyway so I doubt that's the problem.

Fortunately a test download of a story with inline images showed that FFF is getting those okay, so once I get this fixed it shouldn't be a big issue updating the stories I downloaded since whatever went wrong went wrong.

[JimmXinu]

FaceDeer, I may have to disagree with you about your settings.

That story downloaded the cover correctly for me. As did the first three stories off the 'recently approved' list.

I'm using the current test version of FFF.

[FaceDeer]

Interesting. I haven't touched the settings in living memory, but I do have a bunch of customizations from way back. I'll try fiddling them to see what's causing this. Thanks, "It Works On My Machine" is useful here.

[JimmXinu]

Covers are, unfortunately, one of the more complicated bits of FFF.

FYI, I use logger.debug() instead of print() and run in debug mode--I've had print()s not show up, especially in BG jobs.

Also, the cover image for https://www.fimfiction.net/story/501587/ is relatively small (335px × 150px), do you use a cover_min_size setting? (Seems unlikely that would be the case for all your updates, tho.)