[GUI Plugin] FanFicFare

[dennocoil]

1. Yes, that was part of the problem at first.
4. THAT was also the problem since it didn't seem important at first and the documentation confused me on what those extra commands were.
[code]
It works!

[JimmXinu]

So has anyone had any issues with the metadata editing changes?

- Additional metadata edit recursion proofing.
- Allow ==, !=, =~ and !~ in replace_metadata conditionals like in/exclude_metadata.
- Add conditionals_use_lists(default:true) setting for replace_metadata & Include/Exclude metadata conditionals. Might change existing behavior for some users. Set conditionals_use_lists:false in personal.ini if you have problems.

I'd especially like to hear from our users with crazy big/complex metadata editing about if they've noticed any issues.

I'd like to push out an early release this month due to the AO3 changes, but I'm concerned about these changes.

[bballuk]

@JimmXinu

Any chance of getting BoxNovel.com and wuxiaworld.online added to supported sites? I've found another program on github that supports there sites but I like using Calibre and your plugin makes it much easier and quicker.

[fathertyme]

having an odd error this morning. FanFicFare hangs up while trying to update a file. It doesn't error out, it never seems to progress, it just sits at 0% forever

here's the log:

calibre Debug log
calibre 3.37 embedded-python: True is64bit: False
Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.17134')
Python 2.7.12+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 5, 5) && Goodreads (1, 4, 0) && Fantastic Fiction (1, 3, 0) && FanFicFare (3, 3, 9) && Count Pages (1, 8, 2) && Barnes & Noble (1, 2, 15) && Find Duplicates (1, 6, 3) && Generate Cover (1, 5, 21) && Fantastic Fiction Adults (1, 1, 0)
calibre 3.37 embedded-python: True is64bit: False
Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.17134')
Python 2.7.12+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 5, 5) && Goodreads (1, 4, 0) && Fantastic Fiction (1, 3, 0) && FanFicFare (3, 3, 9) && Count Pages (1, 8, 2) && Barnes & Noble (1, 2, 15) && Find Duplicates (1, 6, 3) && Generate Cover (1, 5, 21) && Fantastic Fiction Adults (1, 1, 0)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 96.1893491124 x 96.3318518519
Using calibre Qt style: True
[0.00] Starting up...
[0.02] Initializing db...
[0.09] db initialized
[0.09] Constructing main UI...
FFF: DEBUG: 2019-01-07 00:36:36,996: calibre_plugins.fanficfare_plugin.fff_plugin(207): Plugin FanFicFare macmenuhack file_path:C:\Users\Chad\AppData\Roaming\calibre\pl ugins\fanficfare_macmenuhack.txt
[2.66] main UI initialized...
[2.66] Started up in 2.66 seconds with 1804 books
FFF: DEBUG: 2019-01-07 00:36:48,121: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(41): StoriesOnlineNetAdapter.__init__ - url='https://storiesonline.net/s/18499'
FFF: DEBUG: 2019-01-07 00:36:50,266: calibre_plugins.fanficfare_plugin.fff_plugin(957): FanFicFare v3.3.9
FFF: DEBUG: 2019-01-07 00:36:50,331: calibre_plugins.fanficfare_plugin.fff_plugin(1071) : url:https://storiesonline.net/s/18499
FFF: DEBUG: 2019-01-07 00:36:50,332: calibre_plugins.fanficfare_plugin.config(144): _get_listcache: prefs['rejecturls']
FFF: DEBUG: 2019-01-07 00:36:50,367: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(41): StoriesOnlineNetAdapter.__init__ - url='https://storiesonline.net/s/18499'
FFF: DEBUG: 2019-01-07 00:36:50,374: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(138): URL: https://storiesonline.net/s/18499
FFF: DEBUG: 2019-01-07 00:36:50,374: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1102): #####################################
pagecache(GET) MISS: https://storiesonline.net/s/18499:i
FFF: DEBUG: 2019-01-07 00:36:51,216: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(108): Will now login to URL (https://storiesonline.net/sol-secure/login.php) as (-=-=me=-=-)
FFF: DEBUG: 2019-01-07 00:36:51,217: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1042): #####################################
pagecache(POST) MISS: https://storiesonline.net/sol-secure...rname=XXXXXXXX
FFF: DEBUG: 2019-01-07 00:36:51,660: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(113): HTTP Error 307: Temporary Redirect -- assumed to be valid login for this site
FFF: DEBUG: 2019-01-07 00:36:51,664: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1102): #####################################
pagecache(GET) MISS: https://storiesonline.net/s/18499:i
FFF: DEBUG: 2019-01-07 00:36:52,154: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(200): Number of chapters: 20
FFF: DEBUG: 2019-01-07 00:36:52,176: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1102): #####################################
pagecache(GET) MISS: https://storiesonline.net/a/Lazlo_Zalezac/1
FFF: DEBUG: 2019-01-07 00:36:52,694: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1102): #####################################
pagecache(GET) MISS: https://storiesonline.net/a/Lazlo_Zalezac/2
FFF: DEBUG: 2019-01-07 00:36:53,226: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(280): Found story row on page 2
FFF: DEBUG: 2019-01-07 00:36:53,230: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_storiesonlinenet(366): Do not have a universe
FFF: DEBUG: 2019-01-07 00:36:53,369: calibre_plugins.fanficfare_plugin.fff_plugin(1243) : update existing id:1859
FFF: DEBUG: 2019-01-07 00:36:53,369: calibre_plugins.fanficfare_plugin.fff_plugin(1361) : OVERWRITE file: C:\Users\Chad\OneDrive\Calibre\Fan-Fiction\Lazlo Zalezac\Emend by Eclipse (1859)\Emend by Eclipse - Lazlo Zalezac.epub
FFF: DEBUG: 2019-01-07 00:36:53,369: calibre_plugins.fanficfare_plugin.fff_plugin(1363) : OVERWRITE file updated: 2018-11-12 00:06:42
FFF: DEBUG: 2019-01-07 00:36:53,371: calibre_plugins.fanficfare_plugin.fff_plugin(1368) : OVERWRITE site updated: 2019-01-06 00:00:00
FFF: DEBUG: 2019-01-07 00:36:53,371: calibre_plugins.fanficfare_plugin.fff_plugin(1385) : existing epub tmp:C:\Users\Chad\AppData\Local\Temp\calibre_gwt9m j\fanficfare_kvcy4w\old-1859-kcbpel.epub
FFF: DEBUG: 2019-01-07 00:36:53,450: calibre_plugins.fanficfare_plugin.fff_plugin(1452) : title:Emend by Eclipse
FFF: DEBUG: 2019-01-07 00:36:53,450: calibre_plugins.fanficfare_plugin.fff_plugin(1453) : outfile:C:\Users\Chad\AppData\Local\Temp\calibre_g wt9mj\fanficfare_kvcy4w\Emend by Eclipse-Lazlo Zalezac-fufib_.epub
Exception in thread Thread-3:
Traceback (most recent call last):
File "threading.py", line 801, in __bootstrap_inner
File "site-packages\calibre\utils\ipc\server.py", line 301, in run
File "site-packages\calibre\utils\ipc\server.py", line 220, in launch_worker
CriticalError: Failed to launch worker process:
Traceback (most recent call last):
File "site-packages\calibre\utils\ipc\server.py", line 229, in do_launch
File "site-packages\calibre\utils\ipc\launch.py", line 228, in __call__
File "subprocess.py", line 711, in __init__
File "subprocess.py", line 959, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified

For reference, I did reboot the system twice, I did check the file to make sure it already exists, and I did download and install the newest test code

I'm flummoxed

Thanks

-tyme

[JimmXinu]

@fathertyme -

We had a similar problem reported a month ago which turned out to be overzealous AV. Maybe that's your issue?

@bballuk - I have no interest in adding additional wuxia sites myself at this time. However, I will consider code submitted by others to support them. FanFicFare is an open source project.

[fathertyme]

Originally Posted by JimmXinu

@fathertyme -

We had a similar problem reported a month ago which turned out to be overzealous AV. Maybe that's your issue?

yup.. that was it.. looks like avast stuck one of the calibre executables into the virus chest... took a little reworking, but got it working.. lets see if it remains working!

thnx

[JimmXinu]

New Version Released

Version 3.4.0 - 07 Jan 2019
- Fix defaults.ini add_to_titlepage_entries for hentai-foundry.com
- Fix metadata parsing for adapter_siyecouk
- Fix adapter_whoficcom for site changes.
- Update translations.
- Fix AO3 logout url used to detect when already logged in.
- Add conditionals_use_lists(default:true) setting for replace_metadata & Include/Exclude metadata conditionals. Might change existing behavior for some users.
- Remove outdated comment.
- Allow ==, !=, =~ and !~ in replace_metadata conditionals like in/exclude_metadata.
- Additional metadata edit recursion proofing.
- Allow animated gifs through Calibre Image proc -- need to remove gif.py later.
- Update some comments in adapter_archiveofourownorg
- Additional fix for AO3 login site changes.
- Fix for AO3 login site changes.
- Add bookmarkprivate and bookmarkrec site-specific metadata to adapter_archiveofourownorg

[Atherton25]

First, as always I want to thank the almighty creator Jimmy and all those who have contributed their time to keeping this program running. If I currently wasn't so broke I'd donate money one way or another.

Between vacation and work I've been a bit too busy to catch up on the forums lately, so there are a couple of items here I'll bring up.

@edeniz, just a thought. This may not even be a possibility, but I wanted to put it out there to make sure it's not what's happening. *IF* you have virtual libraries and for some reason filter out some (or all) FFF downloads, when you use FFF whatever is filtered out won't register as being in your library. Because I have such a large library I use several Virtual Libraries (tabs), and I sometimes get duplicates when I'm accidentally in a Virtual Library and I do a download.

Second, a suggestion/thought dealing with wuxiaworld.co. I suggest that m.wuxiaworld.co stuff is accepted and converted to the standard format. the m just is the mobile site, and changing it (automatically) to www is just a matter of switching the m.wuxiaworld.co to www.wuxiaworld.co. It's not that big a deal, but a suggestion.

Last, and I was going to post this far earlier, decided not to, then when I saw the suggestion today decided I'd throw in my support for the two sites. I knew before today that Jimmy is not really interested in adding additional translated webnovel sites *(wuxia/xanxia and the like), since it's been mentioned at least once in the past month, and even further back I think.

I don't want to seem to needy, as FFF is a great program and I hate to ask for more, but it's really damn nice to have the option available to directly download to FFF rather than use webnovelconverter (which hasn't had support in over a year, but still works mostly) or webtoepub (which has a chrome and firefox addon version but doesn't integrate well into calibre).

Anyway, if anyone *is* interested there are two (other than wuxiaworld.co) that are picking up chapters webnovel is hiding behind their paywall setup. While Wuxiaworld.co does cover most of these, it's well behind on several and I've run across some that they do not cover (though I can't list them at the moment). Between these two sites I'm pretty sure everything is covered as far as I can tell.

https://chinesewuxia.world/ ... was actually the one I was going to say, but based on the what I was looking from the link earlier today at it's the same site as https://wuxiaworld.online/ there's a very slight difference that I can see, one gives me a recomendation area while the other doesn't. This may be because I've blocked it in ublock origin though.
https://boxnovel.com/ is of course the other, which was also suggested earlier today.

[JimmXinu]

Originally Posted by Atherton25

First, as always I want to thank the almighty creator Jimmy and all those who have contributed their time to keeping this program running. If I currently wasn't so broke I'd donate money one way or another.

One of the best ways to thank me is... to not call me Jimmy.

Just a pet peeve from childhold. I started using 'jimm' (Jim + last initial) 20 odd years ago when I started a new job and the user name 'jim' was already taken.

Originally Posted by Atherton25

...*IF* you have virtual libraries and for some reason filter out some (or all) FFF downloads, when you use FFF whatever is filtered out won't register as being in your library. Because I have such a large library I use several Virtual Libraries (tabs), and I sometimes get duplicates when I'm accidentally in a Virtual Library and I do a download.

I just tried that and the existing book being hidden in the current VL made no difference--still detected as already in the library.

If you have a series of steps to duplicate the problem, I'd love to hear about it. So far, all I have is hearsay that any problem even exists.

Originally Posted by Atherton25

...m.wuxiaworld.co stuff is accepted and converted to the standard format...

In the next test version.

Originally Posted by Atherton25

Last, and I was going to post this far earlier, decided not to, then when I saw the suggestion today decided I'd throw in my support for the two sites. ...

I (obviously) don't have a problem helping make fanfiction easier to read--neither the authors nor the sites have any great moral high ground to stand on. And I've made it a point to never accept money for FFF or FFDL.

But I worry mildly about the legality of all these Wuxia sites in general. As far as I know, these are translations of published works, not fanfiction, yes? Are the original authors/publishers getting paid for them?

[eschwartz]

Originally Posted by JimmXinu

New Version Released

Version 3.3.0 - 18 Dec 2018

You seem to have listed the version/date for the previous release. At first I thought calibre was showing me an outdated changelog until I triple-checked and saw that there were two 3.3.0 entries.

[Atherton25]

Originally Posted by JimmXinu

One of the best ways to thank me is... to not call me Jimmy.

Just a pet peeve from childhold. I started using 'jimm' (Jim + last initial) 20 odd years ago when I started a new job and the user name 'jim' was already taken.

I don't know why, for some reason, maybe because of the X, even though I know it's JimmX, I think Jimmy. Have since I first was on the boards, I'll try not to make that mistake again.

Originally Posted by JimmXinu

I just tried that and the existing book being hidden in the current VL made no difference--still detected as already in the library.

If you have a series of steps to duplicate the problem, I'd love to hear about it. So far, all I have is hearsay that any problem even exists.

I know it's happened to me several times, (at least 6) I'll see if I can't replicate it. It's usually only a few books since it (generally) has only happened when I use the email update and was using the library for something else before. I'll spend some time on it this week when I have time. It's never really bothered me, just an annoyance that I figured was my mistake, and that's why I suggested maybe that was the problem.

Originally Posted by JimmXinu

In the next test version.

Excellent.

Originally Posted by JimmXinu

I (obviously) don't have a problem helping make fanfiction easier to read--neither the authors nor the sites have any great moral high ground to stand on. And I've made it a point to never accept money for FFF or FFDL.

But I worry mildly about the legality of all these Wuxia sites in general. As far as I know, these are translations of published works, not fanfiction, yes? Are the original authors/publishers getting paid for them?

I obviously don't know the legality of all the sites, I'm not a lawyer, never wanted to be one. Though my cousin is, he's spent so many hours reading such dry crap I don't know if I'd survive.

I'm not sure about the authors getting published for them, but I know there is some setup for some of it having to do with the amount (chapters?) published on some of the sites, I read something about it quite a while back. I don't know if it's just the translator that makes the revenue though, I don't recall if it spoke about how much the original author made for translated work. *But* there is generally some sort of agreement with the authors before it's actually translated - so I'm sure they make some sort of revenue.

However, the original chinese/japanese/korean sites have some pretty draconic rules about who owns the work and how it's published. Plus, as I understand it they generally have a really weird setup that's a combination of number of words (which is why there's a LOT of long stories and lots of repeating the same thing chapter after chapter), visits and frequency of publishing.

[JimmXinu]

Originally Posted by eschwartz

You seem to have listed the version/date for the previous release. At first I thought calibre was showing me an outdated changelog until I triple-checked and saw that there were two 3.3.0 entries.

Fixed. Thanks for pointing that out.

[Iejoor]

Hey,

Firstly thank you for this amazing plugin!

Second, It doesn't work anymore with me. Everytime I try to download something it says it doesn't work. The log gives this: "Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library?"
Does anyone have any idea how to solve this.
Additionally it might be good to know I recently went from windows to linux mint.

[JimmXinu]

Originally Posted by Iejoor

Second, It doesn't work anymore with me. Everytime I try to download something it says it doesn't work. The log gives this: "Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library?"
Does anyone have any idea how to solve this.
Additionally it might be good to know I recently went from windows to linux mint.

You are getting that error while running the FanFicFare plugin in the Calibre GUI???

If so, I wonder if your Calibre install has become corrupted and needs to be re-installed.

That error looks more like something you'd get running FFF as CLI without having all the dependencies installed.

Also, even if you think a problem is happening across the board, please include at least one story URL giving you problems. It still might be a problem only with the site(s) you use.

[Iejoor]

Hey thanks for your reply.
It was indeed solved by removing and then downloading calibre in a different way.
I will keep in mind to add links and such if there's ever a next time.