[GUI Plugin] FanFicFare

[catlover1019]

I'm using the latest stable release, and have noticed what I think is a bug with the "Get Story URLs from Web Page" function. It doesn't seem to work at all on Inkbunny. I always get the same error no matter what I try be it gallery pages, searches or anything. Here's the log from it:

calibre, version 3.45.2
ERROR: Unhandled exception: <b>InvalidStoryURL</b>:Bad Story URL: (https://inkbunny.net/s/=1234567) for site: (inkbunny.net) Example: (https://inkbunny.net/s/=1234567)

calibre 3.45.2 embedded-python: True is64bit: True
Linux-4.18.0-20-generic-x86_64-with-debian-buster-sid Linux ('64bit', 'ELF')
('Linux', '4.18.0-20-generic', '#21~18.04.1-Ubuntu SMP Wed May 8 08:43:37 UTC 2019')
Python 2.7.15
Linux: ('debian', 'buster/sid', '')
Interface language: None
Successfully initialized third party plugins: Count Pages (1, 8, 2) && FanFicFare (3, 10, 0)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 547, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 566, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.getu rls", line 49, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.__init__", line 242, in getAdapter
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.adapter_inkbunnynet", line 43, in __init__
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.base_adapter", line 113, in __init__
InvalidStoryURL: Bad Story URL: (https://inkbunny.net/s/=1234567) for site: (inkbunny.net) Example: (https://inkbunny.net/s/=1234567)

I'm pretty sure it worked just fine a year or so ago, when I last downloaded from Inkbunny.

Note: Everything in the error, including the URLs is exactly the same, no matter what page I'm trying to grab URLs from.

Also, in case anyone was wondering. My previous problems were resolved. Using "Update Epub always" did exactly what I wanted it to.

[JimmXinu]

New Version Released

To address IMAP problems, plus inkbunny because it was small.

Version 3.10.5 - 30 Jul 2019
- Fixes for IMAP folder parsing.
- ensure_str on IMAP4 folder name for Python3. Fixes #419
- Fix adapter_inkbunnynet incorrect example URL, caused problems with get URLs from page.

Version 3.10.0 - 27 Jul 2019
- Additional error checking and output for IMAP4 fetch.
- Add always_include_first_post_chapters to base_xenforoforum_adapter
- XF posts can include a tags without href, look for href searching for links. Closes #417
- Add work around to fix XF2 issue with multiple '...' in threadmarks.
- Tweak getting URLs from email for SV XF2 changes, don't get post URLs.
- Changes to adapter_forumssufficientvelocitycom for XenForo2 migration.
- Fix for Plugin: Group reported results better for meta-only updates.
- Plugin: Group reported results better.
- Add a de-dup check to XF threadmark collection due to at least one SB story having TM bug.
- Adjust reader_posts_per_page for forum.questionablequesting.com.
- adapter_wuxiaworldco: Don't include grayed out 'In preparation' chapters
- Move BS version debug so it doesn't appear in CLI without -d.

[JimmXinu]

Originally Posted by Twilight666

While on Calibre I can install 3.10.5 on CLI the update is to 3.10.1

Updated. Thanks for pointing that out.

[catlover1019]

Thanks for the update. Now, I'm not having that problem any more, but I'm running into something else that I'm not sure is a big, or just intended behavior that's inconvenient.

On Inkbunny, all adult stories, and a lot of other ones are locked down to members-only. I have my login details in personal.ini, and I am able to download these works if I put in the URL. However, the URL getter doesn't seem to be able to get the URLs for these stories, presumably because it doesn't (or can't) login, and these works are completely invisible to guests.

If nothing can be done (or it is just how it's supposed to work), just let me know, and I'll do it the long way, otherwise, I just wanted to let you knoe.

[JimmXinu]

Originally Posted by catlover1019

On Inkbunny, all adult stories, and a lot of other ones are locked down to members-only. I have my login details in personal.ini, and I am able to download these works if I put in the URL. However, the URL getter doesn't seem to be able to get the URLs for these stories, presumably because it doesn't (or can't) login, and these works are completely invisible to guests.

There are a very few of the more popular sites that FFF has special code for in get_urls_from_page() to login/set an adult cookie. I don't really want to try and do it for all sites.

[catlover1019]

Originally Posted by JimmXinu

There are a very few of the more popular sites that FFF has special code for in get_urls_from_page() to login/set an adult cookie. I don't really want to try and do it for all sites.

That's understandable, though inconvenient. Now that I think about it, I'm pretty sure I used something else to gather the URLs before, but I can't remember exactly what.

[LuHallulat]

Hey, I'm sorry, it's possible there's an answer I just haven't figured out the search function well enough to find? But I can't download any fics that are locked to members from archiveofourown.org

During the "fetching metadata" phase, a prompt box pops up asking for my login information. Having entered that, the box pops up again, empty. I enter the info again. At the end of the "fetching metadata" process, it says that the plugin "failed to login", even though it is exactly the same info that logs in successfully on the website.

I have the correct login info entered into the personal.ini box--I have tried removing the hashtags in front of the info, leaving one in front, leaving two in front. None of it changes the above process. Is there something I am doing wrong? Calibre thinks that I have the latest version--3.10.5--is that right?

[Tanjamuse]

Did it work before you updated the plugin?

[JimmXinu]

I'm able to download a users only story entering user/pass on request.

With user/pass and always_login set, it also works fine for me without asking:

[archiveofourown.org]
always_login:true
username:Name
password:Password

My only thought is that when I last logged into AO3 in browser, it asked for a new user-terms agreement. Do you need to do that in browser first?

Yes, v3.10.5 is the current released version because reasons. You can read the thread to see them.

[edeniz]

Just wanted to say: Sorry for not coming back to give feedback after reporting the IMAP problem, but my laptop keyboard died half an hour later and it was a bitch to find replacement.

The new version works perfectly well for me too, no IMAP or other problems. Thank you very kindly for fixing it and for all your hard work!

[JimmXinu]

New Test Version Posted

2019-07-31
- Add XF2 threadmarks_cover/status/desc/title options.

SV (the only XF2 site supported so far) has added some additional 'threadmarks' metadata. I've changed FFF to collect threadmarks_title, threadmarks_description and threadmarks_status as metadata when present.

I've also added settings for whether to use threadmarks_description and threadmarks_status as well as applying the threadmarks cover when present.

These default to on, but are settings because I saw some threads with 'testing' type text in description especially. This way you can turn it off for those, or entirely until it's more mature.

New defaults.ini settings:

[base_xenforo2forum]
## [base_xenforoforum] also applied, but [base_xenforo2forum] takes
## precedence.  SV is the only XF2 site as of Jul 2019.

## Some additional 'thread' metadata entries.
add_to_extra_valid_entries:,threadmarks_title,threadmarks_description,threadmarks_status
#add_to_extra_titlepage_entries:,threadmarks_title,threadmarks_description,threadmarks_status

# Just to remove '_'.
threadmarks_title_label:Threadmarks Title
threadmarks_description_label:Threadmarks Description
threadmarks_status_label:Threadmarks Status

## When use_threadmarks_description is set true,
## threadmarks_description will be used to fill in the standard
## description metadata entry.  Some stories have poor
## threadmarks_description, you can use this setting to avoid using
## it.
use_threadmarks_description:true
## Increasing description_limit from base_xenforoforum's default of
## 500 is also useful with use_threadmarks_description
description_limit:1000

## When use_threadmarks_status is set true, a normalized version of
## threadmarks_status will be used to fill in the standard status
## metadata entry. (In-Progress, Completed plus site statuses.)
use_threadmarks_status:true

## When use_threadmarks_cover is set true, the threadmarks image will
## be used to fill in the cover image.  Set true by default.
use_threadmarks_cover:true

[LuHallulat]

Thank you for responding--I had accidentally deleted the command telling it to login so it had the info but no way to communicate with the site, I guess. Thank you!!

[MerlinMama]

I had to 'Edit Reject URL List' this morning, and while highlighting the required URL and pressing the X button worked fine in removing it from the list, once I hit the OK button, I got the following error and the URL was back in the list.

calibre, version 3.46.0
ERROR: Unhandled exception: <b>TypeError</b>:unsupported operand type(s) for +: 'NoneType' and 'unicode'

calibre 3.46 [64bit]  embedded-python: True is64bit: True
Windows-10-10.0.17763-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.17763')
Python 2.7.15+
Windows: ('10', '10.0.17763', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 6, 3) && Save To Format (0, 0, 6) && Job Spy (1, 0, 162) && Favourites Menu (1, 0, 4) && FanFicFare (3, 10, 5) && Generate Cover (1, 5, 21) && Kobo Utilities (2, 8, 0) && Count Pages (1, 8, 2) && Find Duplicates (1, 6, 3)
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.config", line 673, in show_rejectlist
  File "calibre_plugins.fanficfare_plugin.config", line 202, in add
  File "calibre_plugins.fanficfare_plugin.config", line 152, in _save_list
  File "calibre_plugins.fanficfare_plugin.dialogs", line 151, in to_line
TypeError: unsupported operand type(s) for +: 'NoneType' and 'unicode'

Not something that would be used often, but thought I would mention it. Thanks!

[JimmXinu]

@MerlinMama, I can't reproduce that, but it's easy enough to handle. I've committed the change and it will be included in the next test version.

[MerlinMama]

Originally Posted by JimmXinu

@MerlinMama, I can't reproduce that, but it's easy enough to handle. I've committed the change and it will be included in the next test version.

Hmm, I tried 3 times and checked I had latest version, and same error occurred. Thanks, though!