[GUI Plugin] FanFicFare

[katenepveu]

I'm sure my username and password are correct because I was able to download the archive-locked story, also another random one I just tried. But when I delete those stories and put is_adult:true and always_login:true back, then I get the same error message for the same stories.

I was using a different site skin; but I reverted to the default, and again I was unable to download the above story. Commenting out always_login:true is now appearing to suffice for everything, however.

Sorry! I'm very tired, if this is still being puzzling I will try to do a more useful and thorough rundown of the permutations tomorrow.

[JimmXinu]

@katenepveu - FFF executes the same code when logging in to download a locked/restricted story whether always_login:true or not.

The only way I can think it might make a difference is if you're accidentally effecting username or password in personal.ini

[AleTheLadyInRed]

Originally Posted by JimmXinu

I haven't noticed any issues with images, but I don't read Wattpad and I don't think I read any AO3 stories with images.

If you can provide a few story links and point out some specific chapters, that will help with troubleshooting.

(In fact, I appreciate it if you include at least one story URL whenever reporting issues.)

these are a couple of chapters from different stories that have trouble detecting GIF images:

https://www.wattpad.com/586701525-th...ice-robb-stark

https://www.wattpad.com/601104503-th...ttcon-2018-nyc

(the non-detected GIFs are the ones in text, not the ones at the top of the page - in the first link there's a youtube video on top instead of an image)

for the AO3 links I couldn't remember which story it was (I follow too much of them) but I can assure you there are, in that case they were GIFs from tumblr, maybe I can edit the post when I could find the link again (only SFW right?)

[JimmXinu]

New Test Version Posted

2018-12-29
- Allow animated gifs through Calibre Image proc

Almost four years ago, we had problems with animated gifs in Calibre and the best solution at the time was to simply exclude them.

However, since then the image processing bundled with Calibre has been changed and seems to handle animated gifs better now, so let's try allowing it again.

BTW, if you want the original animated gifs included, you'll need to set no_image_processing:true in personal.ini. eInk readers don't handle animated gifs to the best of my knowledge.

Originally Posted by AleTheLadyInRed

these are a couple of chapters from different stories that have trouble detecting GIF images:

https://www.wattpad.com/586701525-th...ice-robb-stark

https://www.wattpad.com/601104503-th...ttcon-2018-nyc

(the non-detected GIFs are the ones in text, not the ones at the top of the page - in the first link there's a youtube video on top instead of an image)

for the AO3 links I couldn't remember which story it was (I follow too much of them) but I can assure you there are, in that case they were GIFs from tumblr, maybe I can edit the post when I could find the link again (only SFW right?)

[AleTheLadyInRed]

Originally Posted by JimmXinu

New Test Version Posted

2018-12-29
- Allow animated gifs through Calibre Image proc

Almost four years ago, we had problems with animated gifs in Calibre and the best solution at the time was to simply exclude them.

However, since then the image processing bundled with Calibre has been changed and seems to handle animated gifs better now, so let's try allowing it again.

BTW, if you want the original animated gifs included, you'll need to set no_image_processing:true in personal.ini. eInk readers don't handle animated gifs to the best of my knowledge.

I downloaded your latest test version and set "no_image_processing:true" in personal.ini and it works fine using the links I provided earlier. Thank you very much!

Btw I'm using Moon+ Reader on an Android smartphone and over there I can easily see GIFs or other images types in the epub

[DandalfTheWhite]

Apologies if this has been reported already, but I am having issues with Anthologies from AO3. When I go to add a new anthology from a link or try to update an anthology, it causes an error.

Here is an example of updating an anthology fic I created using FanFicFare (this series):

Spoiler:

calibre, version 3.36.0
ERROR: Unhandled exception: <b>HTTPError</b>:HTTP Error 404: Not Found

calibre 3.36 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.17134-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.17134')
Python 2.7.12+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 12, 0) && DeDRM (6, 6, 0) && Package KFX (from KFX Input) (1, 12, 0) && Comicvine (0, 10, 7) && Manage Series (1, 2, 8) && Count Pages (1, 8, 2) && Goodreads (1, 4, 0) && Generate Cover (1, 5, 21) && KFX metadata reader (from KFX Input) (1, 12, 0) && KFX Input (1, 12, 0) && Find Duplicates (1, 6, 3) && DOC Input (1, 0, 1) && Favourites Menu (1, 0, 4) && FanFicFare (3, 3, 0) && Set KFX metadata (from KFX Output) (1, 26, 0) && KFX Output (1, 26, 0) && EpubMerge (2, 4, 0) && Kobo Utilities (2, 6, 0) && KoboTouchExtended (3, 0, 2)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 814, in update_anthology
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 566, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.getu rls", line 66, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.adapter_archiveofourownorg", line 123, in performLogin
File "calibre_plugins.fanficfare_plugin.fanficfare.conf igurable", line 1258, in _postUrl
File "calibre_plugins.fanficfare_plugin.fanficfare.conf igurable", line 1058, in _postUrl
File "urllib2.py", line 435, in open
File "urllib2.py", line 548, in http_response
File "urllib2.py", line 473, in error
File "urllib2.py", line 407, in _call_chain
File "urllib2.py", line 556, in http_error_default
HTTPError: HTTP Error 404: Not Found

And here is an error I get when trying to add a new Anthology from a webpage (this series):

Spoiler:

calibre, version 3.36.0
ERROR: Unhandled exception: <b>HTTPError</b>:HTTP Error 404: Not Found

calibre 3.36 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.17134-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.17134')
Python 2.7.12+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 12, 0) && DeDRM (6, 6, 0) && Package KFX (from KFX Input) (1, 12, 0) && Comicvine (0, 10, 7) && Manage Series (1, 2, 8) && Count Pages (1, 8, 2) && Goodreads (1, 4, 0) && Generate Cover (1, 5, 21) && KFX metadata reader (from KFX Input) (1, 12, 0) && KFX Input (1, 12, 0) && Find Duplicates (1, 6, 3) && DOC Input (1, 0, 1) && Favourites Menu (1, 0, 4) && FanFicFare (3, 3, 0) && Set KFX metadata (from KFX Output) (1, 26, 0) && KFX Output (1, 26, 0) && EpubMerge (2, 4, 0) && Kobo Utilities (2, 6, 0) && KoboTouchExtended (3, 0, 2)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 547, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 566, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.getu rls", line 66, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.adapter_archiveofourownorg", line 123, in performLogin
File "calibre_plugins.fanficfare_plugin.fanficfare.conf igurable", line 1258, in _postUrl
File "calibre_plugins.fanficfare_plugin.fanficfare.conf igurable", line 1058, in _postUrl
File "urllib2.py", line 435, in open
File "urllib2.py", line 548, in http_response
File "urllib2.py", line 473, in error
File "urllib2.py", line 407, in _call_chain
File "urllib2.py", line 556, in http_error_default
HTTPError: HTTP Error 404: Not Found

Regular updating and adding fics from AO3 works fine, and I can add an anthology by copying and pasting the URLs in individually. I just can't use the easy way or update them. It may be something on my end, I just reinstalled Windows and Calibre.

Thanks for the plugin, I love it a lot, and use it all the time. Appreicate the work you do on it.

[JimmXinu]

Originally Posted by DandalfTheWhite

Apologies if this has been reported already, but I am having issues with Anthologies from AO3. When I go to add a new anthology from a link or try to update an anthology, it causes an error.
...

AO3 has changed their login slightly. If you download and install the latest posted test version of the plugin (see link in my sig below), you should be good again.

[DandalfTheWhite]

Oh, awesome. Thank you. I will update to the test version. Appreciate it!

[JimmXinu]

Originally Posted by AmberDragon

I've just run into a small, not super urgent bug while downloading a series. I grabbed "The 7 AUs of Christmas 2018" (https://archiveofourown.org/series/1228217) with the anthology from webpage option and instead of the title being "The 7 AUs of Christmas 2018 Anthology" the title was "Sustained Me (Soulmate Inner Voice) Anthology" - it looks like it pulled the anthology title from the title of the first story instead of from the title of the series. I tried it again after downloading the new test version and the result was the same. Other series I tested downloaded with the proper titles so I don't know what's up with this one. I attached a debug log so you could check it out, but like I said, it's not super urgent, just a little weird.

A further follow up to my previous answer: You could also change the series metadata for that particular story, this post shows an example.

[JimmXinu]

New Test Version Posted (again)

2018-12-29
- Allow animated gifs through Calibre Image proc
- Additional metadata edit recursion proofing.
- Allow ==, !=, =~ and !~ in replace_metadata conditionals like in/exclude_metadata.
- Add conditionals_use_lists(default:true) setting for replace_metadata & Include/Exclude metadata conditionals. Might change existing behavior for some users.

Note: Users with complex replace_metadata / n/exclude_metadata may want to hold off on this one for a bit.

A while back, somebody asked why they couldn't use ==, !=, =~ and !~ in replace_metadata conditionals like in/exclude_metadata. The answer was basically because they were added at different times and I never standardized between them.

I was revisiting that when I also found some other issues. I've fixed a couple more infinite recursion cases, and changed how conditionals work to operate on lists instead of strings so a == conditional will work more intuitively, for example. Otherwise, category='DC','Worm' would not have been matched by &&category==Worm.

However, that change might affect some users, particularly if they are deliberately looking for the list string. For individual cases, you use category_LIST to see the whole list as one string for example.

Or you can get the old behavior back with the setting conditionals_use_lists:false

EDIT:

In/exclude_metadata_pre/post still don't work quite as even I intuitively expect them to. However, I don't think I can 'correct' that without a) writing the whole metadata thing and b) breaking existing behavior for people with complex in/exclude setups. Probably. I'm still thinking about it.

2nd EDIT:

BTW, the impetus for adding ==, !=, =~ and !~ to replace_metadata conditionals is primarily to get !~ for "NOT matching".

The impetus for conditionals_use_lists is so that == will work 'as I expected it to'.

[BookJunkieLI]

So I appear to be having similar problems to katenepveu and DandalftheWhite.

I'm using the latest version of FFF and have version 3.36 of Calibre installed. I've tried downloading the following stories using both my regular login and my secondary login. Neither of them work but I can access the stories through my browser and use AO3's download option. Actually there are at least 22 stories I can't download and all of them are locked/restricted stories.

This is the error message I get when I try downloading them with FFF.

Spoiler:

Status
Title
Author
Comment
URL
Bad
Unknown
Unknown
HTTP Error 404: Not Found
https://archiveofourown.org/works/2413169
Bad
Unknown
Unknown
HTTP Error 404: Not Found
https://archiveofourown.org/works/2406731
Bad
Unknown
Unknown
HTTP Error 404: Not Found
https://archiveofourown.org/works/2383784
Bad
Unknown
Unknown
HTTP Error 404: Not Found
https://archiveofourown.org/works/2299934

[JimmXinu]

New Test Version Posted (again-again)

2018-12-29
- Fix AO3 logout url used to detect when already logged in.
- Allow animated gifs through Calibre Image proc
- Additional metadata edit recursion proofing.
- Allow ==, !=, =~ and !~ in replace_metadata conditionals like in/exclude_metadata.
- Add conditionals_use_lists(default:true) setting for replace_metadata & Include/Exclude metadata conditionals. Might change existing behavior for some users.

Originally Posted by BookJunkieLI

So I appear to be having similar problems to katenepveu and DandalftheWhite.
...

In point of fact, you (BookJunkieLI) were having the first problem, IE, needing the new test version.

But because you included more than one URL in the same download session, I tried doing multiple at once and that revealed the real problem that katenepveu was having. Before this latest version, the test version would work with AO3 again, but only for the first download; subsequent would fail.

Should be fixed now.

[BookJunkieLI]

Ah, I'd assumed the version available directly through Calibre was the latest.
I downloaded the test version from your link and all is well now! Thanks for getting it working again so quickly!

[edeniz]

I've had some strange behaviour with fanficfare lately:

I choose "update epub if new chapters", but fanficfare seems to interpret that as "add new book". I would add a debug report here, but I keep forgetting to start calibre in debug mode. I couldn't find any sort of consistency for this behaviour anyway, so I'm at a loss. (Could it have anything to do with fanficfare getting the updates from email inbox?)

Basically what I'm trying to do with this post is to ask if anyone else has had this experience Does anyone else suddenly have a plethora of duplicate epubs? And if it's possible for the "update if new chapters" and the "add new book" options of fanficfare to have such a mix up.

[JimmXinu]

Originally Posted by edeniz

I've had some strange behaviour with fanficfare lately:
...

Somebody reported a similar sounding issue on the google group, but didn't have enough details to find anything.

I didn't hear anything further after suggesting:

Since I'm not seeing it and I'm not hearing anyone else having an issue, all I can think of is to reset the "Default If Story Exists" setting in case it was changed--that is, actually change it and change it back to make sure it's set.

Or reinstalling on the off chance your copy of the plugin was corrupted.

Is it happening when you add stories manually by URL, or by IMAP fetch?
If IMAP, do you have the email notice that resulted in the duplicate?
If you mark it unread will it happen again?

What site(s)? Are they exactly exactly identical by identifiers:url?
If so, does an further update by URL attempt fail with "More than one identical book by Identifier URL"?

What language do you use with calibre?