[GUI Plugin] FanFicFare

[JimmXinu]

@Tanjamuse - We don't know what the criteria are for 'too much' and they're going to be different for different sites. So while slow_down_sleep_time gives a measure of protection from being blocked, it's not a guarantee.

At this time, I'm not interested in adding more mass downloading capability to the plugin.

[Tanjamuse]

That's okay. I just wanted to ask.

[JimmXinu]

Originally Posted by Montana Harper

FIRST ISSUE FIXED! (I had an errant override that was keeping it hidden.)

Progress!

Originally Posted by Montana Harper

For the second issue:
...
I can use include/exclude metadata to get rid of any extraneous stuff, so that's not a problem. I just need to know if there's a way to move individual items from one entry name to another (maybe using regex in an if/then kind of way, like replace_metadata, but more comprehensive).

Normalizing metadata from different sites is always going to be an uphill battle. But because an eFiction site(like wraithbait) generally only has so many 'tags', it's not a sisyphean task like it is on AO3.

This is how I would approach it:

[www.wraithbait.com]
## First, you may as well use the same metadata names.  
## Then you can use [defaults] to set them to cust columns.
## (ships is always defined)
add_to_extra_valid_entries:,freeformtags,ao3categories

## make sure you can see the changes on titlepage for testing:
add_to_titlepage_entries:,freeformtags,ao3categories,ships

## second, put a copy of the data you want in each:
## '.NOREPL' tells the system to *not* apply the source metadata's
## in/exclude/replace_metadata -- Only works on include_in_ lines.
## That prevents problems with recursion loops.
include_in_ships:category.NOREPL
include_in_ao3categories:category.NOREPL
include_in_freeformtags:category.NOREPL,genre.NOREPL

## then use include and exclude to limit what's in each.
## (https://github.com/JimmXinu/FanFicFare/wiki/InExcludeMetadataFeature)
## (since I had to look it up myself)
## pre vs post will only matter if you have replace_metadata lines that
## operate on ships, ao3categories or freeformtags.
include_metadata_post:
# assume ships (and only ships) have '/' in them.
 ships=~/
 ships==Other Pairings
 ships==Other Male Pairings
 ao3categories==Slash Pairings
 ao3categories==Ship Pairings
 ao3categories==Threesomes and Moresomes

exclude_metadata_post:
# presumably, exclude everything that is included in ships or ao3categories
 freeformtags=~/
 freeformtags=~Pairings
 freeformtags==Threesomes and Moresomes

This works for me, with the two examples given at least. You can add more lines or conditionals to the include/excludes to fine tune as needed.

[Montana Harper]

Originally Posted by JimmXinu

This works for me, with the two examples given at least. You can add more lines or conditionals to the include/excludes to fine tune as needed.

Perfect! I tweaked it a little for purposes of replace_metadata (normalizing the pairings, etc.) and it works exactly like I wanted it to. Once I see it laid out, it makes perfect sense, too, so I should be able to extrapolate for most other sites I might want to download from in the future.

Thank you so much!

[JimmXinu]

Originally Posted by Montana Harper

Thank you so much!

[Montana Harper]

*sneaks back into forum*

One more tiny thing? Is there a way to use replace_metadata to change one piece of metadata into two comma-separated pieces? On Wraithbait, they use "Bitextual" and I want to swap it to the AO3 format of M/M, F/M.

Example story: http://www.wraithbait.com/viewstory.php?sid=6284

If not, I'll leave it as Bitextual and use Calibre's built in search & replace periodically to clean it up.

Thanks!

[JimmXinu]

Originally Posted by Montana Harper

One more tiny thing? Is there a way to use replace_metadata to change one piece of metadata into two comma-separated pieces? On Wraithbait, they use "Bitextual" and I want to swap it to the AO3 format of M/M, F/M.

Example story: http://www.wraithbait.com/viewstory.php?sid=6284

.....

Okay, whatever.

It took me entirely too long to remember that there's a feature that does exactly that. Add \, to the replacement string.

add_to_replace_metadata:
 category=>Bitextual=>M/M\,F/M

(update)BTW, I'm adding some documentation for that feature to defaults.ini and the wiki page.

[Montana Harper]

Originally Posted by JimmXinu

Add \, to the replacement string.

I could've sworn I'd already tried escaping the comma, but I guess I must not have, because it works now.

Thanks again.

[Blackedout]

Originally Posted by JimmXinu

New test version posted (direct link).

Nothing horribly pressing, just some tweaks I found to fix/improve.

2015-10-11
Fixes for base_xenforoforum #post- URLs -- might effect other site URLs with # anchors.

Account for base href in XenForo so emoticon images work. - Use [epub] nook_img_fix:false to show emoticon images in line.

Correct cover_exclusion_regexp for base_xenforoforum.

Fix for unNew with & in chapter title.

Fixes/New Features for test1.

So, is it possible that this update caused it to no longer recognize questionable questing urls, because all of the ones I just tried to put in were recognized as valid urls.

[JimmXinu]

Originally Posted by Blackedout

So, is it possible that this update caused it to no longer recognize questionable questing urls, because all of the ones I just tried to put in were recognized as valid urls.

It's possible. Give me some of the URLs that failed/where invalid.

[Blackedout]

Originally Posted by JimmXinu

It's possible. Give me some of the URLs that failed/where invalid.

Here are 4 of them, they didn't even show up when quoted and starting download.

https://forum.questionablequesting.c...or-story.1414/
https://forum.questionablequesting.c...oc.2349/unread
https://forum.questionablequesting.c...o-x-mgq-e.503/
https://forum.questionablequesting.c...st.2314/unread

[JimmXinu]

Originally Posted by Blackedout

Here are 4 of them, they didn't even show up when quoted and starting download.

Thanks.

More problems caused by the fact that QQ used questionablequesting.com when I first added it and now uses forum.questionablequesting.com. I'm done playing around with trying to support both names, so from now on FFF will only recognize forum.questionablequesting.com.

You will likely need to change your personal.ini section name, too:

[forum.questionablequesting.com]

Test version posted (direct link)

2015-10-14
Update wiki link in defaults.ini.

Tweak to epub logpage that helps with finding desc.

Improve replace_metadata comments in defaults.ini

Change default HTML output to HTML5 header.

Update html tag header at beginning of each epub file. Prompted by failure of old header on latest tolino ereader.

Accept only forum.questionablequesting.com, not questionablequesting.com.

[CakeChan]

Hello. I use the command-line version of FFF on a tablet with a reader that doesn't support horizontal rule tags, so I'm using the "replace_hr: true" option but it's not converting the tags. I recall this used to work in the past.

The bulk of the stuff I'm getting with FFF is from FF.net and AO3, in epub. The issue started with just FF.net about a year ago... AO3 downloads used to work fine but now they also don't convert.

Is this a bug, an issue with the site's coding, or a mistake with my config somehow?

Thanks so much.

[JimmXinu]

Originally Posted by CakeChan

Hello. I use the command-line version of FFF on a tablet with a reader that doesn't support horizontal rule tags, so I'm using the "replace_hr: true" option but it's not converting the tags. I recall this used to work in the past.
...

That's a bug.

We're (slowly) switching over to a newer HTML parser and hr tags come out as <hr/> instead of <hr_/> (where _ is really a space). And the replace_hr feature used a straight text replacement for <hr_/>. I've changed it to a regex instead, so it will work for both in future.

You say you use CLI. Unfortunately, I don't have a good way to release test versions of CLI. But there should be a new release in less than a week.

[tricklem]

Originally Posted by JimmXinu

I haven't. Testing with a Gmail account, only UNSEEN messages are grabbed and messages deleted or marked deleted aren't included in the UNSEEN list.

I would speculate that either your email client isn't deleting the messages on the server right away when you tell it to, or if you are using a different email provider it may include messages marked deleted in UNSEEN.

What email client and provider are you using?

I am using a Hotmail account.