[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by Ryune

Well fuck. Xenforo is borked again... They are doing a new appending of the indexes that collapses it down to 8 links. You can expand it by clicking on the ellipsis but FFF can't read it at the moment.

An example.

This true.

It's so true that Xon (the Xenforo guy) told me about it.

I hope to have a test fix some time tonight or tomorrow.

Originally Posted by Ryune

EDIT: also, Royal Road is now using links going to RoyalRoad.com instead of RoyalRoadL.com Adding an L to the link still works fine apparently but FFF doesn't see it as a valid link without the L.

Changes already in the latest test version.

[Ryune]

Originally Posted by JimmXinu

This true.

It's so true that Xon (the Xenforo guy) told me about it.

I hope to have a test fix some time tonight or tomorrow.



Changes already in the latest test version.

Thanks for the prompt fixes in that case. I always forget that the buglist is there >.< sorry.

[JimmXinu]

Originally Posted by Ryune

Thanks for the prompt fixes in that case. I always forget that the buglist is there >.< sorry.

No problem. You did, in fact, post before the issue was submitted on github.

I'll have a test update tomorrow.

[JimmXinu]

New Test Version Posted

2018-09-04
- Restore cursor in finally: clauses in case of issues.(plugin)
- Update Translations
- Fix for base_xenforoforum (SB/SV specifically) change to 'hide' sections of threadmark lists behind '...'. Issue #332
- Fix empty Genre parsing for adapter_ficwadcom
- Fix genre parsing for adapter_fanficauthorsnet.

[JimmXinu]

New Test Version Posted

2018-09-05
- Fix adapter_harrypotterfanfictioncom date and characters/genre splitting.
- Fix adapter_fanficauthorsnet metadata parsing and genre splitting.
- Add --no-meta-chapters/-z CLI option.

[JimmXinu]

New Test Version Posted

2018-09-07
- Fix FimF login.

This is going to be last test version before the next release (unless new problems arise).

[eschwartz]

In celebration of the python3 port, I think I will move the CLI version to Arch Linux's [community] repository (currently maintained by me in the AUR).

For any Arch users, now is your chance to vote for the package and convince me this is in high demand.

[chicleeblair]

Hi,

I’m incredibly new to this—I’ve been using Fanficfare for a while, but really doubled down on the features in the past couple of days.

I didn’t realize until now how many Marvel and Marvel-adjacent fandoms I read in, and also how many different ways AO3 categorizes them. I want to roll all of the Captain America fics into one category, and for whatever reason, I can’t manage to make the most basic replace metadata category work. Could someone give me an example using Captain America (2011) and Captain America - All Media Types that I can adapt for the numerous other varieties?

[JimmXinu]

Originally Posted by eschwartz

In celebration of the python3 port, I think I will move the CLI version to Arch Linux's [community] repository (currently maintained by me in the AUR).

Cool. But I'm going to point anyone asking Arch installation specific questions to you, then.

FYI, I'm going to bump the version to v3.0 when I release it, early next week if nothing else comes up.

Originally Posted by chicleeblair

I didn’t realize until now how many Marvel and Marvel-adjacent fandoms I read in, and also how many different ways AO3 categorizes them. I want to roll all of the Captain America fics into one category, and for whatever reason, I can’t manage to make the most basic replace metadata category work. Could someone give me an example using Captain America (2011) and Captain America - All Media Types that I can adapt for the numerous other varieties?

There's an FAQ on ReplaceMetadata, but here's a bit to get you started.

The first thing to realize is that long as AO3 lets anyone enter anything for metadata, you're unlikely to get all of it wrangled exactly the way you all the time without manual intervention.

But you can get a lot of it with a bit of work.

The most straightforward way is to explicitly replace the tags you run across in your stories:

[archiveofourown.org]
add_to_replace_metadata:
 category=>Captain America - All Media Types=>Captain America
 category=>Captain America \(Movies\)=>Captain America

But you end up with a lot of lines if you start down that path, especially as you apply them to more fields and fandoms and you have to escape regex characters like () as shown.

Or you can use regular expressions.
For example, if you just want every category name that contains 'Captain America' to be changed to 'Captain America', this will do it:

[archiveofourown.org]
add_to_replace_metadata:
 category=>^.*Captain America.*$=>Captain America

Translation of regex:

• ^ = start of string
• .* = 0 or more of any character
• Captain America = Captain America
• .* = 0 or more of any character
• $ = end of string

That particular pattern isn't perfect. It won't match, for example:

• Capt America
• Captian America
• captain america
• CaptainAmerica

And it will match, for example:

• Old Avengers Not Captain America
• Captain American Ship Hornet WWII

If you have specific stories or ini problems, it's best to include story URLs and a copy of the relative ini lines--if not your whole personal.ini--there's a button to get a 'safe' copy with usernames and passwords removed.

[chicleeblair]

Thank you so much!

AO3 is actually pretty regulated in terms of Category/Fandom, they just are also liberal with subcategories/cross-posts. There's a lot of Captain American (YEAR) - Media Type, for instance, or double-posting on A Song of Ice and Fire AND Game of Thrones. I did, at least, figure out how to deal with this myself. (It was a matter of setting the fandom columns to "text separated by commas," not simply text, if anyone else is sometimes oblivious like me.)

[eschwartz]

Originally Posted by JimmXinu

Cool. But I'm going to point anyone asking Arch installation specific questions to you, then.

We actually prefer that. If users have a belief that their problem might be installation-related and thus distro-related, they should *not* be asking upstream authors to help with things that work fine on that end... and for something I use a lot myself, I can often catch bug handling from both sides.

Anyway, you know my Github username, I'm very responsive to notifications there (less so on MobileRead, but I do get email notifications for PMs).

FYI, I'm going to bump the version to v3.0 when I release it, early next week if nothing else comes up.

Version bump patterns are nothing I cannot handle, thanks for the heads up on your timeline.

[chicleeblair]

Okay, the consolidation of Captain America worked like a dream, and I was able to duplicate it for several other fandoms! Yay! But now I've been picking at the code again for so long that my dog is practically asking to go to bed, so I thought I'd put my question out into the world before signing off for the night. I'm sure it's a matter of a backslash or a quotation mark, but oh well.

I'm pulling from two older archive sites, one of which is thechosentwo, and i imagine this will be the same with different names). The use the "category" tag for more than fandom, and chosen two's information doesn't even populate my ship column. however, I can't figure out how to just overwrite all their metadata in those categories and replace it.

I've tried variations, but basically what I've used so far is replace_metadata and include_metadata_pre like so:

[chosentwofanfic.com]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true
include_metadata_pre:
 category==Buffy the Vampire Slayer
replace_metadata:
 category=>Buffy the Vampire Slayer
 ships=>Faith Lehame/Buffy Summers

I assume if this worked I'd probably be using one or the other, but like I said no variation I tried worked.

[JimmXinu]

Originally Posted by chicleeblair

I'm pulling from two older archive sites, one of which is thechosentwo, and i imagine this will be the same with different names). The use the "category" tag for more than fandom, and chosen two's information doesn't even populate my ship column. however, I can't figure out how to just overwrite all their metadata in those categories and replace it.

include_metadata_pre doesn't add metadata, it creates a white list of what metadata will be allowed.

In truth, defaults.ini should include extracategories:Buffy the Vampire Slayer for [chosentwofanfic.com]. I'll add that before the next release.

replace_metadata lines like category=>Buffy the Vampire Slayer are two element lines, so category is taken as the pattern to be replaced. See wiki here.

You can also add characters & ships for all stories from a site like so:

[chosentwofanfic.com]
extracategories:Buffy the Vampire Slayer
extracharacters:Buffy, Faith
extraships:Buffy/Faith

But not all stories from chosentwofanfic.com are Buffy/Faith. You might instead want something like this:

[chosentwofanfic.com]
extracategories:Buffy the Vampire Slayer

# put category into ships before replacement is done (.NOREPL)
include_in_ships:category.NOREPL

# put characters into ships after replacement is done.
include_in_characters:ships

include_metadata_pre:
# Only keep ships that have a /
 ships=~/
# Only keep the one category (from extracategories)
 category==Buffy the Vampire Slayer

add_to_replace_metadata:
# split charA/charB into a list of charA, charB
 characters=>([^/]+)/(.+)=>\1\,\2

[JimmXinu]

New Version Released

Version 3.0.0 - 10 Sep 2018
- adapter_chosentwofanficcom: use pagecache and extracategories:Buffy the Vampire Slayer
- Fix FimF login.
- Fix adapter_harrypotterfanfictioncom date and characters/genre splitting.
- Fix adapter_fanficauthorsnet metadata parsing and genre splitting.
- Add --no-meta-chapters/-z CLI option.
- Restore cursor in finally: clauses in case of issues.(plugin)
- Update Translations
- Fix for base_xenforoforum (SB/SV specifically) change to 'hide' sections of threadmark lists behind '...'. Issue #332
- Fix empty Genre parsing for adapter_ficwadcom
- Fix genre parsing for adapter_fanficauthorsnet.
- Tweak mobi output--move TOC to end.
- Remove some CLI debug output (conflist).
- Improve CLI --save-cache, save on each fetch/post, fix a py2/py3 cross bug.
- Small fix for <> appearing in text format.
- Remove defunct site www.area52hkh.net
- Remove defunct site www.fiction.thebrokenworld.org
- Remove defunct site dramione.org
- Remove defunct site writing.whimsicalwanderings.net
- Remove defunct site www.destinysgateway.com
- Remove defunct site imrightbehindyou.com
- Remove defunct site nocturnal-light.net
- Remove defunct site national-library.net
- Add latestonly option to mark_new_chapters feature to remove pre-existing (new) chpater marks on update and only mark chapters that are new in this update. Closes #330
- Attempting to get password protected stories in FimF working again.
- Fix fanfiction.net metadata parsing -- genre was ending up in language after BS change.
- Fix mobi output -- Fix an encoding issue, Kindle Goto-TOC works, TOC links are more accurate
- Accept both [royalroad.com] and pre-existing [royalroadl.com] - Change [royalroadl.com] to [www.royalroad.com]
- royalroadl.com has changed to www.royalroad.com.- Update included_dependencies
- Many, many changes to make code run on both python 2.7 and python 3.

[edeniz]

Hello,

I've got a metadata problem that's driving me bonkers.

I've got a replace_metadata textfile that is several thousand lines long (which is why I'm not attaching it here). Plus I've set the sleep_down_time as default to six. (Though I use whatever's in the default ini if I'm in a hurry.) So it's not unusual for it to take far longer than it should to download a link from sites like AO3. Especially the download metadata part takes long. I'm used to it and even expect it at this point. Not like I'm usually in any sort of hurry.

That said, there is one link that makes calibre simply freeze at the downloading metadata stage. (I once tried to wait it out, but had to force close on calibre after 2 hours.)

The link in question:

http://archiveofourown.org/works/8892235[1]

Trying to hunt down the culpable line on the replace_metadata file is proving difficult. Especially since whatever the issue, is only happening with *this* link, and no other. And because when I think I might have found the faulty section by way of elimination (as in: deleting several sections, trying to download to see if it helped so I know where to look more closely.), it turns out false alarm (as in: when I have only that section in the replace_metadata file, the download still works out fine somehow).

Some time ago, a new feature was added to fanficfare whereby sometimes the error messages tell us exactly which line of the metadata is hindering the download. Which makes me wonder just what it takes to freeze calibre up without such a message now, and in the download metadata stage at that.

What I've tried:

* escaped all non-alpha-numeric except \ just to be sure
* checked matching ^ and $
* checked if all periods in tags are escaped
* checked if ( and ) in the tags are escaped

My track record is such that I'm 100% sure that the answer is glaringly obvious, that whatever mistake there is in the metadata, I keep coming across it and overlooking it.

So I thought I'd ask for a tip just what I should be on the lookout for. Has this happened to anyone before? What else could I check aside from the above mentioned? Is there a good way to hunt down the faulty line that I don't know of?

Can you help me out here?


Thank you kindly in advance!