[GUI Plugin] FanFicFare

[fanfictioneer]

Originally Posted by JimmXinu

New Test Version Posted

2019-01-15
- Add always_login setting to base_xenforo for SV login-required story with 404 result.


I've never seen a user/pass required story on SV before.

It's the first time I have ever seen it, and I wonder if it is something the thread starter can toggle or if it is a mod-only ability. Any who, thanks a bunch for fixing it!

By the way, could not a workaround be to try login if you receive a 404 (on xenforo)? I mean, generally speaking people don't try to download stories by just mashing on keys to generate the URL, so it stands to reason that one could spend the extra time taken to log in in that particular case?

[JimmXinu]

Originally Posted by fanfictioneer

By the way, could not a workaround be to try login if you receive a 404 (on xenforo)? I mean, generally speaking people don't try to download stories by just mashing on keys to generate the URL, so it stands to reason that one could spend the extra time taken to log in in that particular case?

Yes, and I coded that first. But then it would stop and ask you your user/pass 2-3 times before accepting it's a real 404.

[JimmXinu]

New Test Version Posted

2019-01-18
- Fix author parsing at webnovelcom - pull request #372 from Rikkitp
- Remember original href in data-orighref attr with epub internalize_text_links so inserted 'earlier' chapters don't break internal links.

[sandmanwake]

Originally Posted by fanfictioneer

It's the first time I have ever seen it, and I wonder if it is something the thread starter can toggle or if it is a mod-only ability. Any who, thanks a bunch for fixing it!

It's the Unlisted Fiction part of SV, which is similar to the NSFW section of QQ in that it requires you to log-in to view. It says so near the top of the page when you first click on the Unlisted Fiction link:

"Unlisted Fiction

Only viewable by logged in users. For when you want to keep something off the published nets, say if you don't want to give away your right of first sale."

Edit:
Note that Unlisted Fiction is a subforum of the User Fiction forum and depending on your settings, the threads under Unlisted Fiction may show up along with all the regular threads when you're viewing the User Fiction section and logged in. If you click on a thread that was posted under Unlisted Fiction, it'll take you to that subforum and the particular thread.

Edit2:
SB also has a section of unlisted fiction that you can only see if you sign in.

[chrnno]

Been having trouble with SV lately all threads I tried give HTTP Error 503: Service Temporarily Unavailable, no other site I use has any problem including SB/QQ which are largely similar(EDIT: Tested AH too and it works). Lately I have had Cloudfare annoy me with waiting when I am accessing the site(I think only when I haven't in a while not sure) so might be related?

calibre Log de Debug
calibre 3.38.1 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.15+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: pt_BR
Successfully initialized third party plugins: EpubMerge (2, 5, 0) && Count Pages (1, 8, 2) && Generate Cover (1, 5, 21) && FanFicFare (3, 4, 0) && Find Duplicates (1, 6, 3)
calibre 3.38.1 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.15+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: pt_BR
Successfully initialized third party plugins: EpubMerge (2, 5, 0) && Count Pages (1, 8, 2) && Generate Cover (1, 5, 21) && FanFicFare (3, 4, 0) && Find Duplicates (1, 6, 3)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 71.9842323651 x 71.9822878229
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.09] splash screen shown
[0.09] Initializing db...
[1.14] db initialized
[1.14] Constructing main UI...
EpubMerge: DEBUG: 2019-01-21 23:26:47,125: calibre_plugins.epubmerge.epubmerge_plugin(154): macmenuhack file_path:C:\Users\Felipe\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
FFF: DEBUG: 2019-01-21 23:26:47,140: calibre_plugins.fanficfare_plugin.fff_plugin(207): Plugin FanFicFare macmenuhack file_path:C:\Users\Felipe\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
[6.22] main UI initialized...
[6.22] Hiding splash screen
[49.91] splash screen hidden
[49.91] Started up in 49.91 seconds with 5022 books
FFF: DEBUG: 2019-01-21 23:27:42,441: calibre_plugins.fanficfare_plugin.fff_plugin(957): FanFicFare v3.4.0
FFF: DEBUG: 2019-01-21 23:27:42,543: calibre_plugins.fanficfare_plugin.fff_plugin(1071): url:https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: INFO: 2019-01-21 23:27:42,543: calibre_plugins.fanficfare_plugin.prefs(206): Attempting to read settings from predecessor--FFDL
FFF: INFO: 2019-01-21 23:27:42,543: calibre_plugins.fanficfare_plugin.prefs(211): Using default settings
FFF: DEBUG: 2019-01-21 23:27:42,545: calibre_plugins.fanficfare_plugin.config(144): _get_listcache: prefs['rejecturls']
FFF: INFO: 2019-01-21 23:27:42,687: calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_xenforoforum_adapter(324): url: https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: DEBUG: 2019-01-21 23:27:42,687: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1102): #####################################
pagecache(GET) MISS: https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: DEBUG: 2019-01-21 23:27:49,954: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1102): #####################################
pagecache(GET) MISS: https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: DEBUG: 2019-01-21 23:28:00,608: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1102): #####################################
pagecache(GET) MISS: https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: DEBUG: 2019-01-21 23:28:16,305: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1102): #####################################
pagecache(GET) MISS: https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: DEBUG: 2019-01-21 23:28:23,055: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1181): Giving up on https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/
FFF: DEBUG: 2019-01-21 23:28:23,056: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1182): HTTP Error 503: Service Temporarily Unavailable
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.fanficfare.configurable", line 1170, in _fetchUrlOpened
  File "calibre_plugins.fanficfare_plugin.fanficfare.configurable", line 1129, in _fetchUrlRawOpened
  File "urllib2.py", line 435, in open
  File "urllib2.py", line 548, in http_response
  File "urllib2.py", line 473, in error
  File "urllib2.py", line 407, in _call_chain
  File "urllib2.py", line 556, in http_error_default
HTTPError: HTTP Error 503: Service Temporarily Unavailable
FFF: ERROR: 2019-01-21 23:28:23,056: calibre_plugins.fanficfare_plugin.dialogs(708): Exception: {u'uniqueurl': u'https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/[None-None]', u'comment': u'HTTP Error 503: Service Temporarily Unavailable', u'good': False, u'publisher': None, u'site': 'forums.sufficientvelocity.com', u'begin': None, u'calibre_id': None, u'added': False, u'showerror': True, u'title': u'Unknown', u'author': [u'Unknown'], u'listorder': 0, u'url': u'https://forums.sufficientvelocity.com/threads/fragmentation-original-superhero-setting.25437/', u'collision': u'Update EPUB if New Chapters', u'series': u'', u'author_sort': [u'Unknown'], u'pubdate': None, u'comments': u'', u'end': None}:HTTP Error 503: Service Temporarily Unavailable
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 696, in do_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1156, in prep_download_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1038, in get_story_metadata_only
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter", line 287, in getStoryMetadataOnly
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter", line 371, in doExtractChapterUrlsAndMetadata
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_xenforoforum_adapter", line 327, in extractChapterUrlsAndMetadata
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_xenforoforum_adapter", line 103, in _fetchUrlOpened
  File "calibre_plugins.fanficfare_plugin.fanficfare.configurable", line 1282, in _fetchUrlOpened
  File "calibre_plugins.fanficfare_plugin.fanficfare.configurable", line 1183, in _fetchUrlOpened
HTTPError: HTTP Error 503: Service Temporarily Unavailable

[JimmXinu]

Originally Posted by chrnno

Been having trouble with SV lately all threads I tried give HTTP Error 503: Service Temporarily Unavailable, no other site I use has any problem including SB/QQ which are largely similar(EDIT: Tested AH too and it works). Lately I have had Cloudfare annoy me with waiting when I am accessing the site(I think only when I haven't in a while not sure) so might be related?

Something similar happened to somebody else a page back.

The user_agent setting is about the only thing that can be changed in FFF that might make a difference.

[chrnno]

Originally Posted by JimmXinu

Something similar happened to somebody else a page back.

The user_agent setting is about the only thing that can be changed in FFF that might make a difference.

...Somehow despite reading through the page since I saw SV mentioned I completely missed that between the threadmark and login only ones

Oh well at least the rest still work and I have all the ones I got previously even if I can't update the active ones.

Thank you for the quick answer.

[curiouscat]

Hi, love the plugin JimmXinu. Can you please checkout 'Merton.com' and it's companion board 'A Happy Assembly' Some great Jane Austen fiction on there.

[JimmXinu]

Originally Posted by curiouscat

Hi, love the plugin JimmXinu. Can you please checkout 'Merton.com' and it's companion board 'A Happy Assembly' Some great Jane Austen fiction on there.

Merton.com is a real estate site... Story URLs?

[megme]

I think curiouscat meant http://meryton.com/

[JimmXinu]

I don't think meryton.com is a good candidate.

It's a forum site that doesn't have threadmarks or anything else that could be used as a chapter list.

Plus, it requires user/password for everything and has non-automatic user creation. Which still doesn't allow access to other archival parts of the site.

[cryzed]

From what I remember reading in the changelog when it was released the Python3 version (despite having a different code) is equaly supported as the Python2 version and has the exact same functionalities and results. Right?

If you are talking about FanFicFare, yes the Python 2 and Python 3 version should be functionally equivalent.

And since at some point I have to replace Python2 with Python3 is it worth the effort to do so now? And if I choose to wait and even not do so at all, what are your plans?

Calibre's developer is a bit stubborn about the port to Python 3, he'd rather maintain a custom Python 2 fork himself, assuming that that is less work. FanFicFare's codebase should work with both Python 2 and 3 in stand-alone mode.

[Tanjamuse]

When I download a story from FFnet I can see how many a story has been clicked favorite or simply hit. Is it possible to do something like that in Calibre as well?

To show me how many times I've read a story?

[JimmXinu]

Multi-reply

Originally Posted by Twilight666

Quick question.
Lately the CLI has been showing me warnings about Python2 no longer being supported after 1/1/2020. ...

Short answer: If you use FFF CLI and don't specifically need Py2 for something else, I'd probably switch to Py3 now.

That warning is no doubt coming from your Python2 executable, since it likely comes from the official Python org.

FFF uses the same code on Py2 & Py3, but with some 'translation' code(a version of six.py) for when the two are incompatible.

I have every expectation that somebody will continue to release Python2 after the official Python org stops. Odds are, there will be more than one; it's all open source, after all. However, getting, installing and maintaining a Py2 install is all outside scope of FFF.

FFF will continue to run on Py2 as long as Calibre continues to be Py2.

Originally Posted by cryzed

Calibre's developer is a bit stubborn about the port to Python 3, he'd rather maintain a custom Python 2 fork himself, assuming that that is less work. FanFicFare's codebase should work with both Python 2 and 3 in stand-alone mode.

While making FFF work on both Py2 & Py3, I was shocked at the number of version incompatibilities I saw between the two that I could think of no reason for other than purely philosophical disagreements. I got the feeling the Py3 committee didn't like the Py2 developers and were going to show them how it should done (EDIT: In their(Py3's) opinion). But I might be reading too much into it.

Regardless, Py3 skirts the line between being a new version and a different language, IMHO. I don't blame Kovid for maintaining a Py2 version since he's already been doing that for years for other reasons.

Originally Posted by Tanjamuse

When I download a story from FFnet I can see how many a story has been clicked favorite or simply hit. Is it possible to do something like that in Calibre as well?

To show me how many times I've read a story?

I'm not aware of anything other adding a custom column and maintaining it yourself. I suggest you search and/or ask in the Library Management forum. It's a bit outside the scope of FFF.

[curiouscat]

Originally Posted by megme

I think curiouscat meant http://meryton.com/

Thanks I did