[GUI Plugin] FanFicFare

[edeniz]

Originally Posted by JimmXinu

Copying that snippet into my personal.ini, I don't see any issues with the one story given.

replace_metadata isn't supposed to allow infinite loops--and even if it did, Calibre would run out of memory and error before hours had passed.

If you can send me a copy of your full personal.ini file it might help (use the 'View "Safe" personal.ini' button).

If you use the "Saved Metadata Column" feature (which I recommend to anyone with complex metadata setup), you can prove whether it's the metadata processing or not by doing an update with "Update Calibre Metadata from Saved Metadata Column" option.

You could also run Calibre in debug mode and see if you get any useful logging from that.

so much! I'll send you the file. But first, I'm gonna try your suggestion about the saved metadata column and then do a debug, so you have as much info as possible.

[JimmXinu]

New Test Version Posted

2018-12-04
- Improve recursion-proofing of replace_metadata for performance and duplicate lines.
- Don't cache file: URLs. Option --save-cache can mask changes to _filelist files while debugging.
- Don't sleep when fetching file: URLs.
- Remove defunct site www.artemis-fowl.com
- Remove defunct site asr3.slashzone.org
- Remove defunct site tolkienfanfiction.com
- Remove defunct site fanfiction.tenhawkpresents.com
- Remove defunct site unknowableroom.org
- Remove defunct site mujaji.net
- Update adapter_harrypotterfanfictioncom for site change.

@edeniz - This version finishes processing your insane replace_metadata in (only!) a few minutes (3-5) for me.

The lines you identified were duplicated, which lead me to improve recursion-proofing.

I'm not dead sure if that duplication was the problem, or if it's just the complexity explosion from having thousands of replace_metadata and hundreds of exclude lines. But this version seems to work.

[edeniz]

Originally Posted by JimmXinu

New Test Version Posted

2018-12-04
- Improve recursion-proofing of replace_metadata for performance and duplicate lines.
- Don't cache file: URLs. Option --save-cache can mask changes to _filelist files while debugging.
- Don't sleep when fetching file: URLs.
- Remove defunct site www.artemis-fowl.com
- Remove defunct site asr3.slashzone.org
- Remove defunct site tolkienfanfiction.com
- Remove defunct site fanfiction.tenhawkpresents.com
- Remove defunct site unknowableroom.org
- Remove defunct site mujaji.net
- Update adapter_harrypotterfanfictioncom for site change.

@edeniz - This version finishes processing your insane replace_metadata in (only!) a few minutes (3-5) for me.

The lines you identified were duplicated, which lead me to improve recursion-proofing.

I'm not dead sure if that duplication was the problem, or if it's just the complexity explosion from having thousands of replace_metadata and hundreds of exclude lines. But this version seems to work.

, Jim!

Can you tell me where it was duplicated so I can delete those lines?

Yeah, insane is a bit of an understatement, isn't it?

But then, I've been using fanficfare for years at this point, and I'm just a tad bit OCD about the ugly tagging systems of archives.

(Of course if the AO3 authors stopped using the tags as author's notes, they'd do us all a favour, but alas, they won't.)

[JimmXinu]

Originally Posted by edeniz

, Jim!

Can you tell me where it was duplicated so I can delete those lines?

Yeah, insane is a bit of an understatement, isn't it?

But then, I've been using fanficfare for years at this point, and I'm just a tad bit OCD about the ugly tagging systems of archives.

(Of course if the AO3 authors stopped using the tags as author's notes, they'd do us all a favour, but alas, they won't.)

One copy was at the bottom. The other was somewhere in the middle. I did a comparison and those were the only duplicated lines.

[edeniz]

Originally Posted by JimmXinu

One copy was at the bottom. The other was somewhere in the middle. I did a comparison and those were the only duplicated lines.

Thanks again!

[JimmXinu]

New Test Version Posted

2018-12-05
- New Site: www.hentai-foundry.com - Stories only.

In case it isn't obvious from the domain name, this is very much a NSFW site -- it doesn't just have explicit content, it's for explicit content. You are warned.

(I've made sure to not make the domain a link, let me know if that's not sufficient not-linking.)

[regina3264]

Hi, I apologize in advance for what is almost certainly a very stupid question (I am new to this forum, I'm sorry) but since the latest update of FFF a few days ago, I haven't been able to use it to download any fic. I've tried a fanfiction.net link and then double-checked with an ao3 link just to make sure the problem wasn't on the part of ff.net. Neither works at all.

It's not returning any error messages - it just runs as an endless job. It used to say "Downloading" and just show endlessly at 1% forever until I manually quit the job, and then I rolled back Calibre to the previous install, thinking that might have been the problem, and now it just shows "Waiting" with the status as "Unavailable," but it still just runs and runs like that until I force a stop.

I've tried everything I can think of short of rolling back FFF itself, and that's only because I can't figure out how to do that. Is there a place I can get previous downloads? (I apologize again if this is super obvious, apparently I have no idea what I'm doing.)

[JimmXinu]

Originally Posted by regina3264

...
I've tried everything I can think of short of rolling back FFF itself, and that's only because I can't figure out how to do that. Is there a place I can get previous downloads? (I apologize again if this is super obvious, apparently I have no idea what I'm doing.)

Uninstalling and reinstalling FFF is what I would recommend.

The Plugins Introduction thread explains in detail, but go to Calibre's Preferences -> Plugins, search for FanFicFare, use the Remove button. Restart Calibre.

Then either install it again using the usual 'Get Plugins' mechanism, or download it manually from the first post and install from Preferences -> Plugins -> Load Plugin from File.

There's a good chance that will fix it for you.

If not, try running Calibre in debug mode from Calibre's Preferences drop-down and see if you gives you any messages.

[regina3264]

Originally Posted by JimmXinu

Uninstalling and reinstalling FFF is what I would recommend.

The Plugins Introduction thread explains in detail, but go to Calibre's Preferences -> Plugins, search for FanFicFare, use the Remove button. Restart Calibre.

Then either install it again using the usual 'Get Plugins' mechanism, or download it manually from the first post and install from Preferences -> Plugins -> Load Plugin from File.

There's a good chance that will fix it for you.

If not, try running Calibre in debug mode from Calibre's Preferences drop-down and see if you gives you any messages.

Thanks for the reply. Uninstalling and reinstalling was one of the things I had tried before, but I tried it again just to be sure. That didn't fix it, so I ran Calibre in debug, and...it's definitely telling me something, I just...don't know how to read it. Does this by chance mean anything to you?

FFF: DEBUG: 2018-12-09 19:26:13,188: calibre_plugins.fanficfare_plugin.fanficfare.adapt ers.adapter_fanfictionnet(100): URL: https://www.fanfiction.net/s/13016156/1/
FFF: DEBUG: 2018-12-09 19:26:13,190: calibre_plugins.fanficfare_plugin.fanficfare.confi gurable(1099): #####################################
pagecache(GET) MISS: https://www.fanfiction.net/s/13016156/1/
FFF: DEBUG: 2018-12-09 19:26:17,217: calibre_plugins.fanficfare_plugin.fff_plugin(1250) : from URL(https://www.fanfiction.net/s/13016156/1/)
FFF: DEBUG: 2018-12-09 19:26:17,601: calibre_plugins.fanficfare_plugin.fff_plugin(1452) : title:Being Harry Potter
FFF: DEBUG: 2018-12-09 19:26:17,601: calibre_plugins.fanficfare_plugin.fff_plugin(1453) : outfile:C:\Users\USER\AppData\Local\Temp\calibre_y ta4ut\fanficfare_j7sh6k\Being Harry Potter-Kirinin-w8fgz7.epub
Exception in thread Thread-3:
Traceback (most recent call last):
File "threading.py", line 801, in __bootstrap_inner
File "site-packages\calibre\utils\ipc\server.py", line 301, in run
File "site-packages\calibre\utils\ipc\server.py", line 220, in launch_worker
CriticalError: Failed to launch worker process:
Traceback (most recent call last):
File "site-packages\calibre\utils\ipc\server.py", line 229, in do_launch
File "site-packages\calibre\utils\ipc\launch.py", line 228, in __call__
File "subprocess.py", line 711, in __init__
File "subprocess.py", line 959, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified

[JimmXinu]

Originally Posted by regina3264

Thanks for the reply. Uninstalling and reinstalling was one of the things I had tried before, but I tried it again just to be sure. That didn't fix it, so I ran Calibre in debug, and...it's definitely telling me something, I just...don't know how to read it. Does this by chance mean anything to you?

...
CriticalError: Failed to launch worker process:
Traceback (most recent call last):
  File "site-packages\calibre\utils\ipc\server.py", line 229, in do_launch
  File "site-packages\calibre\utils\ipc\launch.py", line 228, in __call__
  File "subprocess.py", line 711, in __init__
  File "subprocess.py", line 959, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified

That looks like an error launching the Calibre background process. FFF was functioning correctly before that.

That suggests that there's something wrong with your Calibre install maybe, or PATH. I'd tell you to reinstall Calibre, but it sounds like you already did. Are you able to launch other background processes, such as Calibre's Convert?

Google search offers a few ideas, the most recent of which suggests it could be your AV being over-zealous.

If none of that helps, we'll have to ask Kovid.

[regina3264]

Originally Posted by JimmXinu

That looks like an error launching the Calibre background process. FFF was functioning correctly before that.

That suggests that there's something wrong with your Calibre install maybe, or PATH. I'd tell you to reinstall Calibre, but it sounds like you already did. Are you able to launch other background processes, such as Calibre's Convert?

Google search offers a few ideas, the most recent of which suggests it could be your AV being over-zealous.

If none of that helps, we'll have to ask Kovid.

Oh, wow, you are a hero! That worked! My AV program was the culprit. Thank you so much - I was getting so frustrated!

[JimmXinu]

New Test Version Posted

2018-12-10
- Update translations
- Workaround for adapter_hentaifoundrycom bad dateUpdated value.

Also added comment to defaults.ini about:

[www.hentai-foundry.com]
## Sometimes (perhaps often?) HF stories word counts and size are
## incorrect.  You can exclude them from being used like so:
add_to_exclude_metadata_pre:
 numWords=~.*
 size=~.*

[shscs911]

Can you please add boxnovel.com support for FFF plugin?

[GadGray]

Would it be possible to add support for HFY subreddit using the wiki index.

edit: not all have the story in just the first post (continuing in the comments by the author)


also, you update alot and quickly. That is remarkable.

[Elaine T]

My kid and I on different computers, browsers and versions of Calibre are failing to download anything from AO3. I tried uninstalling the plugin and re-installing, and nothing changed.
The symptoms are not helpful: nothing happens. No job starts. If I try the menu option "get story downloads from URL' I get the error 'no valid Story URLS found on given page.' Even when I'm looking at the AO3 story on the page.

help? The kid really hates having to use AO3's download facility.