[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by bugstomper

Although, what about a general purpose adaptor for situations like this...

There are already other tools out there for creating ebooks from web pages as a general thing.

I'm not interested in supporting more general solutions in FFF because I rather doubt that it's going to be as close as you think, honestly.

[bugstomper]

Originally Posted by JimmXinu

'm not interested in supporting more general solutions in FFF because I rather doubt that it's going to be as close as you think, honestly.

It looks like you are right about that. I now have two serial books I'm reading that are on wordpress.com. They are both just a tiny bit of extending basic_adaptor and even easier to do the second one after I did the first. But I don't see how I would generalise anything else to make anything easier for a non-programmer.

However, is there any way to make it so I don't have to manually unzip the plugin, add in my adaptors and modified __init__.py, and zip it up again every time Calibre updates the plugin? Perhaps something that could go in personal.ini to specify paths of custom adaptors or specify a directory containing them?

[JimmXinu]

Originally Posted by bugstomper

However, is there any way to make it so I don't have to manually unzip the plugin, add in my adaptors and modified __init__.py, and zip it up again every time Calibre updates the plugin? Perhaps something that could go in personal.ini to specify paths of custom adaptors or specify a directory containing them?

When I re-designed the internal structure several years ago, I tried really hard to set it up so all that was required was having adapter source files in a certain directory (inside the zip, for plugin) with a common name pattern.

I was never able to come up with a way that worked across all three FanFicFare platforms (Calibre plugin, web service and python CLI). Admittedly, I was new with Python then and I'm still no classloader expert now.

Plus, even if I could I'm not sure it would be a good idea to allow loading of arbitrary code for security reasons.

Your best bet right now is to keep your code in your own github fork, merge my upstream changes into that periodically and install that version in your calibre. This does require git and learning a least a bit about how to use it.

Roughly:

1. Make a github account
2. Create a fork of the FanFicFare project on github (there's a 'fork' button)
3. Clone your fork to a working directory on your Calibre machine (follow github instructions)
4. Set an upstream remote to the project (git remote add upstream https://github.com/JimmXinu/FanFicFare.git)
5. Create (and checkout) a branch in your working directory (somewhat optional)
6. Make your changes in your working directory and commit them
7. From your working directory, run FFF's makeplugin.py
8. Install the plugin zip in Calibre with calibre-customize -a FanFicFare.zip
9. Then, when FanFicFare updates, instead of installing normally, go to your working directory, merge upstream changes with git fetch upstream; git merge upstream/master then repeat steps 7 & 8.

[sonofden]

Having some problems with xen forums (spacebattles and sufficient velocity).


It downloads the book but only the first post, despite the fact there are several threadmarked chapters. Here are examples of problem stories:

https://forums.spacebattles.com/thre...ssover.544303/

https://forums.sufficientvelocity.co...aracter.39758/


It is not all stories on these websites, just some. Any help or guidance would be appreciated.


Edit: Nevermind. For some reason some times the plugin picks up the wrong address. When I manually correct the url and re-download it works.

[ClairePMR]

Hi Jimm - dramione.org is up and operating again. Would very much appreciate having it added back in! Thanks!

[JimmXinu]

New test version posted

2017-07-17
- Restore adapter_dramioneorg after site came back.

[ILB]

I get an error message (IndexError:list index out of range) if I want to use "Get Story URLs from Web Page"
example http://archiveofourown.org/series/558469

Spoiler:

calibre, version 3.4.0
ERROR: Unhandled exception: <b>IndexError</b>:list index out of range

calibre 3.4 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.15063-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.15063')
Python 2.7.12+
Windows: ('10', '10.0.15063', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Generate Cover (1, 5, 21) && Count Pages (1, 8, 0) && Copy Cover To Device (2, 0, 0) && EpubMerge (2, 1, 1) && Quick Preferences (1, 4, 1) && FanFicFare (2, 14, 1) && Similar Stories (1, 0, 58) && Find Duplicates (1, 6, 3) && Reading List (1, 6, 6) && SmartEject (2, 0, 7) && Manage Series (1, 2, 8) && Kindle Collections (1, 7, 17)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 548, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 568, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.getu rls", line 61, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.adapter_archiveofourownorg", line 106, in performLogin
IndexError: list index out of range

[Loba]

Originally Posted by ILB

I get an error message (IndexError:list index out of range) if I want to use "Get Story URLs from Web Page"
example http://archiveofourown.org/series/558469

Getting the same when trying create anthologies and also when updating existing ones. Error below.

Spoiler:

calibre, version 3.3.0
ERROR: Unhandled exception: <b>IndexError</b>:list index out of range

calibre 3.3 Portable embedded-python: True is64bit: False
Windows-10-10.0.15063-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.15063')
Python 2.7.12+
Windows: ('10', '10.0.15063', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: EpubMerge (2, 1, 1) && SmartEject (2, 0, 7) && View Manager (1, 4, 3) && Reading List (1, 6, 6) && EpubSplit (2, 2, 1) && Favourites Menu (1, 0, 4) && Generate Cover (1, 5, 21) && FanFicFare (2, 14, 0) && Quick Preferences (1, 4, 1) && Kindle Collections (1, 7, 17) && Prettify Cover (0, 9, 9) && Find Duplicates (1, 6, 3) && Count Pages (1, 8, 0) && Consolidate All Library Metadata (2, 0, 32)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 810, in update_anthology
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 568, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.getu rls", line 61, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.adapter_archiveofourownorg", line 106, in performLogin
IndexError: list index out of range

Ao3 did a deploy last night.

Originally Posted by Ao3_Status Twitter

The Archive is now running on Rails 4.2, with further upgrades in the works. Issues in the previous deploy have been resolved as well.

[JimmXinu]

AO3 has changed their login mechanism a bit, breaking FFF in the process. I'm looking into it.

If you comment out / set fale always_login under [archiveofourown.org], you should at least be able to download stories that don't require login.

[JimmXinu]

New test version posted

2017-07-17 again
- Fix for AO3 login change.

2017-07-17
- Restore adapter_dramioneorg after site came back.

[JimmXinu]

New Test version posted

2017-07-20
- Fix for images from some sources ending up with // accidentally in longdesc.
- Fixes for replace_br_with_p causing extra tags when run more than once on the same text. Now adds <!-- FFF_replace_br_with_p_has_been_run --> markers to prevent it.
- Add replace_failed_smilies_with_alt_text option for base_xenforo.
- Add some debug perf prof output.
- Fix published and update date for gravitytales.com adapter - import feedparser (#204) from cryzed

replace_failed_smilies_with_alt_text details:

Spoiler:

## SV & SB (and possibly QQ and AH) like to use sprite smilies --
## smiley images that are all in one image file for performance and
## are shown by CSS background with offsets.  Epub viewers don't like
## that.
##
## The sprite smiley img tags generally do have alt attributes that get
## the point across: ":)" ":D" ":(" ":lol" etc.  And not all smilies
## on these sites use sprite images, so some can be used as is.
##
## When replace_failed_smilies_with_alt_text:true, any images that
## failed to download or are src=clear.png *and* have a class containing
## mceSmilie and an alt attribute will be replaced with:
## <span class="(classes)">(alt text)</span>
##
## The smiley classes are preserved in case the user wants to do
## something special with them.
#replace_failed_smilies_with_alt_text:false

[JimmXinu]

Something I learned today: FanFicFare already has code in it for a user custom icon.

See the end of the discussion on this issue.

[Loba]

Re:AO3 login changes

I'm getting the following error when always_login is set to true on the latest test version. Different error than before, but always login still has to be commented out.

'NoneType' object has no attribute '__getitem__'

[JimmXinu]

I'm not seeing it--I can download with user/pass fine on AO3.

I suggest you install the test version again and make sure you restart calibre afterwards.

Originally Posted by Loba

Re:AO3 login changes

I'm getting the following error when always_login is set to true on the latest test version. Different error than before, but always login still has to be commented out.

'NoneType' object has no attribute '__getitem__'

[Loba]

Originally Posted by JimmXinu

I'm not seeing it--I can download with user/pass fine on AO3.

I suggest you install the test version again and make sure you restart calibre afterwards.

Hmmm I must have fudged up the install somehow. I restarted a few times after installing it. Idek. Working fine now. Thanks.

Edit:
Nevermind. It's still happening.

Status
Title
Author
Comment
URL
Error
With A Little Christmas Magic
AceLotti
'NoneType' object has no attribute '__getitem__'
https://archiveofourown.org/works/590407

That's the full copypasta. I've tried updating both fics that require login and ones that don't. I must have clicked a non-ao3 fic after reinstalling. Works just fine when I comment out always_login:true for fics that don't require login. I'll try reinstalling again, full restart pc and all that and report back if I get it to work. *confused*