[GUI Plugin] FanFicFare

[JimmXinu]

(Multi-reply)

Originally Posted by shscs911

Can you please add boxnovel.com support for FFF plugin?

I, personally, don't have any interest right now in adding support for more novel translation sites.

Originally Posted by GadGray

Would it be possible to add support for HFY subreddit using the wiki index.
...

Reddit is not a good fit structurally for FFF. I won't say it's impossible, but I'm not interested in doing it.

Originally Posted by Elaine T

My kid and I on different computers, browsers and versions of Calibre are failing to download anything from AO3.
...

I'm not seeing any problems downloading from AO3. And if it was a general problem for all users more people would be reporting it.

What you're describing sounds like you may not be using correct story URLs. What URLs are you seeing the problem with? (You want the 'Download from URLs' option, usually.) If you run Calibre in debug mode (Calibre Preferences dropdown), do you get more information?

Sometimes AV/firewall software causes problems while opening HTTP connections from programs like Calibre, but those usually manifest as connection failures.

Another possibility is that you may have generated so much traffic in a relatively short amount of time that AO3 is blocking or throttling your downloads. I don't recall proving that conclusively with AO3 before, but I've suspected it before and other sites have done it. And again, that usually manifests as connection failures.

[Elaine T]

I've run Calibre in debug mode and don't see anything obvious to me in the log, but maybe someone else will. I gave it 8 URLs to download (these: https://archiveofourown.com/works/11261703
https://archiveofourown.com/works/13...iew_adult=true
https://archiveofourown.com/works/13155444
https://archiveofourown.com/series/673037
https://archiveofourown.com/users/te...s/teaandtumblr
https://archiveofourown.com/works/12657768
https://archiveofourown.com/works/11585544
https://archiveofourown.com/works/12886884)
all of which the kid and I have tried to download singly, together and in various combinations.

The Calibre debug log says:

calibre Debug log
calibre 3.34 embedded-python: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '7', '6.1.7601')
Python 2.7.12+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 5, 5) && EpubMerge (2, 4, 0) && FanFicFare (3, 2, 0)
calibre 3.34 embedded-python: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '7', '6.1.7601')
Python 2.7.12+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 5, 5) && EpubMerge (2, 4, 0) && FanFicFare (3, 2, 0)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 72.035450517 x 72.0
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.06] splash screen shown
[0.06] Initializing db...
[0.23] db initialized
[0.23] Constructing main UI...
EpubMerge: DEBUG: 2018-12-13 13:35:51,973: calibre_plugins.epubmerge.epubmerge_plugin(152): macmenuhack file_path:C:\Users\Elaine\AppData\Roaming\calibre\ plugins\fanficfare_macmenuhack.txt
FFF: DEBUG: 2018-12-13 13:35:51,982: calibre_plugins.fanficfare_plugin.fff_plugin(207): Plugin FanFicFare macmenuhack file_path:C:\Users\Elaine\AppData\Roaming\calibre\ plugins\fanficfare_macmenuhack.txt
[4.32] main UI initialized...
[4.32] Hiding splash screen
[85.77] splash screen hidden
[85.77] Started up in 85.77 seconds with 1992 books
FFF: DEBUG: 2018-12-13 13:37:47,838: calibre_plugins.fanficfare_plugin.fff_plugin(957): FanFicFare v3.2.0

[JimmXinu]

Originally Posted by Elaine T

I've run Calibre in debug mode and don't see anything obvious to me in the log, but maybe someone else will. I gave it 8 URLs to download (these: https://archiveofourown.com/works/11261703
...

FFF recognizes archiveofourown.org, not archiveofourown.com.

I don't recall seeing archiveofourown.com before. I need to investigate a bit before deciding if FFF will starting supporting it and how.

UPDATE:
BTW, these aren't valid story URLs, even with .org. You can get story URLs from them, but they aren't stories themselves.
https://archiveofourown.com/series/673037
https://archiveofourown.com/users/te...s/teaandtumblr

[JimmXinu]

New Test Version Posted

2018-12-13
- Accept archiveofourown.com for archiveofourown.org. archiveofourown.org remains the 'canonical' domain.

archiveofourown.com and archiveofourown.org are both owned by transformativeworks.org. But archiveofourown.org is what's linked from transformativeworks.org, so it remains canonical.

[Elaine T]

Thanks. the .com, .org just never registered to our eyes. And when we put them the browser the story pages showed stories, so we couldn't figure out what was wrong.
live and learn.

[JimmXinu]

New Version Released

Version 3.3.0 - 18 Dec 2018
- Accept archiveofourown.com for archiveofourown.org. archiveofourown.org remains the 'canonical' domain.
- Update translations
- Workaround for adapter_hentaifoundrycom bad dateUpdated value.
- New Site: www.hentai-foundry.com - Stories only.
- Improve recursion-proofing of replace_metadata for performance and duplicate lines.
- Don't cache file: URLs. Option --save-cache can mask changes to _filelist files while debugging.
- Don't sleep when fetching file: URLs.
- Remove defunct site www.artemis-fowl.com
- Remove defunct site asr3.slashzone.org
- Remove defunct site tolkienfanfiction.com
- Remove defunct site fanfiction.tenhawkpresents.com
- Remove defunct site unknowableroom.org
- Remove defunct site mujaji.net
- Update adapter_harrypotterfanfictioncom for site change.
- Change for adapter_storiesonlinenet for 'Full Access' stories.
- base_xenforoforum_adapter - Fix for buggy threadmarks on SV thread 46020.

[theoddpassenger]

On AO3 you have the ability to mark a bookmark as a 'Private Bookmark' or a 'Rec' with checkboxes when saving a bookmark. I looked through the INI and didn't see it referenced and it doesn't appear to be pulled when I pull the bookmark tags. Is this metadata that FFF currently pulls? If not, is that something that could possibly be added?

[JimmXinu]

Originally Posted by theoddpassenger

On AO3 you have the ability to mark a bookmark as a 'Private Bookmark' or a 'Rec' with checkboxes when saving a bookmark. I looked through the INI and didn't see it referenced and it doesn't appear to be pulled when I pull the bookmark tags. Is this metadata that FFF currently pulls? If not, is that something that could possibly be added?

That's not a feature I use, so I don't know if it's still working, but it used to.

You need to have always_login:true set, however. From defaults.ini:

[archiveofourown.org]
## Some sites require login (or login for some rated stories) The
## program can prompt you, or you can save it in config.  In
## commandline version, this should go in your personal.ini, not
## defaults.ini.
#username:YourName
#password:yourpassword

## In order to get bookmarktags and bookmarksummary, you need to login
## all the time.  This defaults to off to save time and network
## traffic.  Requires valid AO3 username and password when true.
#always_login:false

[theoddpassenger]

Originally Posted by JimmXinu

That's not a feature I use, so I don't know if it's still working, but it used to.

Hm, good to know, thanks! I've got always_login set to true and have custom columns for bookmarktags and bookmarksummary that are working, although it doesn't show up in either of those. Do I need to set up a new custom column for it? I don't see anything else in the extra_valid_entries list that appears to match.

[JimmXinu]

I went back and found my one bookmarked story and tested it, and it both collected the metadata entries and put them in custom columns when configured to do so using custom_columns_settings.

So it's working as far as I can tell.

[theoddpassenger]

Originally Posted by JimmXinu

I went back and found my one bookmarked story and tested it, and it both collected the metadata entries and put them in custom columns when configured to do so using custom_columns_settings.

Sorry if I'm misunderstanding. I'm sick and possibly not comprehending stuff as well as I should be! You're saying you have custom columns set up that show the Private Bookmark & Rec flags? Can I ask what custom column configuration is showing them? I've got the below, which is working for the tags & summary, but doesn't grab the check box values from what I can see.

custom_columns_settings:
 bookmarktags=>#bookmarktags,a
 bookmarksummary=>#bookmarksummary

I was looking through the AO3 adapter on github (assuming the plugin uses the same adapters) and see the part that's pulling the bookmark_tag_string and bookmark_notes data, but I didn't see any reference to the bookmark_private or bookmark_rec checkbox values.

[JimmXinu]

Okay, I see that I'm the one who has been misunderstanding.

You are correct that there is code for bookmarktags and bookmarksummary--that's what I thought we were talking about. I didn't realize "Private Bookmark & Rec flags" were separate settings on a bookmark.

After a little research, it looks like I could probably collect the current settings of those flags.

But what you want them for? Completionist impulses? They're your settings...

And yes, the GitHub repository is the code base I use and update.

[theoddpassenger]

Originally Posted by JimmXinu

But what you want them for? Completionist impulses? They're your settings...

I mean, you're not wrong! 😂 I use them on AO3 to help me filter my bookmarks so I can sort of have multiple levels/tiers of bookmarks and it would be convenient to be able to filter them in Calibre the same way. I could use bookmark tags to achieve the same thing (it would probably actually make more sense!) but that would require me to go back and edit all my existing bookmarks 😅.

[JimmXinu]

New Test Version Posted

2018-12-22
- Add bookmarkprivate and bookmarkrec site-specific metadata to adapter_archiveofourownorg

Because what's 2 more site-specific metadata entries when there's 18 to start with.

These values are True or False for stories that you have bookmarked and empty for stories you have not bookmarked.

[theoddpassenger]

Originally Posted by JimmXinu

2018-12-22
- Add bookmarkprivate and bookmarkrec site-specific metadata to adapter_archiveofourownorg

I super appreciate it.