[GUI Plugin] FanFicFare

[eschwartz]

Originally Posted by JimmXinu

Interesting. I see that issue, too. The normalization of the series entry must happen farther upstream than the 'mi'(MetaInformation) object. The code for calling the GC plugin already re-fetches the updated DB 'mi', it's pretty trivial to do that before the calibre gen cover too.

Thanks for pointing that out.

[readerboy7]

Once again, I am experiencing a weird and rare issue with replace_br_with_p: true, on FFF v 2.2.17, this time when downloading from FFnet to epub format (I haven't tried other formats yet)
Specifically, when converting chapter 7 of https://www.fanfiction.net/s/345691/1/Girl-Days . Chapters 1 to 6, and all the rest work perfectly fine, but for some reason, chapter 7 fails. Only chapter 7.

Originally Posted by chapter 7

It was a beautiful morning. Just beautiful. The sun rose like

the very eye of God, the sky was an intense sky-blue (a good thing,

since if it had been an intense sky-green we'd have an entirely

different sort of fanfic, wouldn't we?) and the breeze from the

mile-away beach held the tang of the ocean.

And Ranma couldn't decide which swimsuit to wear.

In contrast to chapter 1, and the other chapters that I'm not quoting from:

Originally Posted by chapter 1

Ranma Saotome scowled as she walked down the streets of Nerima. "Someday," she snarled, "I'm gonna find out HOW that ladle-lapping old coot always hit's me. She NEVER misses. It's like some wet version of the tomboy's hammer..."

As she entered the courtyard of the Tendo Dojo, she failed to notice the person standing in the house's doorway. Ranma stopped just in time to keep from bumping to her--

Mother.

Strangely enough, I looked at the source code for both chapters, and there was no major difference in formatting.

It was a beautiful morning. Just beautiful. The sun rose like<br>the very eye of God, the sky was an intense sky-blue (a good thing,<br>since if it had been an intense sky-green we'd have an entirely<br>different sort of fanfic, wouldn't we?) and the breeze from the<br>mile-away beach held the tang of the ocean.<br><br> And Ranma couldn't decide which swimsuit to wear.

Ranma Saotome scowled as she walked down the streets of Nerima.<br>"Someday," she snarled, "I'm gonna find out HOW that<br>ladle-lapping old coot always hit's me. She NEVER misses. It's<br>like some wet version of the tomboy's hammer..."<br><br> As she entered the courtyard of the Tendo Dojo, she failed to<br>notice the person standing in the house's doorway. Ranma stopped<br>just in time to keep from bumping to her--<br><br> Mother.<br><br>

This is confusing, and quite weird.

[JimmXinu]

Originally Posted by readerboy7

Once again, I am experiencing a weird and rare issue with replace_br_with_p: true, on FFF v 2.2.17, this time when downloading from FFnet to epub format (I haven't tried other formats yet)
...

replace_br_with_p is a heuristic or 'best effort' feature. It's never going to be perfect. And I believe it was designed to replace double <br> tags between paragraphs with <p> tags, not to unwrap lines.

I suggest investigating Calibre's convert unwrap feature (Under 'Heuristic Processing'). Calibre's Edit Book may have something for that now too.

Heck, if you want, I'll give you a copy of my hand-fixed epub of that story.

The ffnet date on Girl Days is 2001 and I think I remember reading it back in the 90s. Not quite back in the script-format era, but close enough to recall it. I don't expect it's going to be changing much.

[readerboy7]

Nah, don't worry. I found a copy online split into txt files (still with line splits) and I once wrote a program in python 3.4 to download files in that format (with some work on my part*), and I can have calibre convert it to my preferred formats. Deals with line splits fine, it has.
But thanks for the offer.

* Basically, giving it the main webpage and the list of txt files it has to download

[davidfor]

Originally Posted by JimmXinu

Okay. I can code it to look for Incomplete and Inactive and set status to be something like:

• Incomplete and Inactive
• Incomplete
• Abandoned

Opinions? My inclination is to go with Incomplete and Inactive except that other statuses are single word.

As the site uses it, I tend to "Incomplete and Inactive". And for a single word "Incomplete". For some reason, I just don't like "Abandoned".

[fathertyme]

Originally Posted by fathertyme

Interesting. This implies that it's just me that is having this issue. I'll have to investigate further to try and figure it out on my end.

So I threw a couple debug lines into the adapter to check, and status is getting set, but not showing up when the book is done.

Which means its probably a setting I've screwed up somewhere... I'll keep looking, but it's almost definitely not an adapter issue.

Any ideas where to look?

or why it is not affecting "Completed" but just "In-Progress" ?

(I'll figure it out, just hoping there might be some bread crumbs laying around to follow)

[JimmXinu]

New test version (direct link)

2016-02-10
Adding Incomplete status state to adapter_storiesonlinenet
Add new AccessDenied exception for one line output in CLI.
Allow Pairing w/o Centered category -> chars in TtH.

[JimmXinu]

Originally Posted by fathertyme

So I threw a couple debug lines into the adapter to check, and status is getting set, but not showing up when the book is done.

Which means its probably a setting I've screwed up somewhere... I'll keep looking, but it's almost definitely not an adapter issue.

Any ideas where to look?

I'd start with trying it in a separate test library that doesn't have your usual personal.ini settings. Then copy parts over testing between each to isolate the issue.

The replace_metadata or include/exclude lines are where I would look first.

[readerboy7]

Is there a way to get FFF to only download certain chapters of stories, when downloading fics? I'd imagine it'd make de-bugging faster (e.g. when the problem only appears on certain chapters, and you don't need to download the whole fic), or if you want to split one fic into multiple books, and it'd be more efficient than using calibre's edit book function.

[JimmXinu]

Originally Posted by readerboy7

Is there a way to get FFF to only download certain chapters of stories, when downloading fics?

Hover your mouse cursor over the edit box you enter story URLs in. The tooltip explains that you can put [2,4] for example after the URL to only download chapters 2, 3 and 4. Or [4] for just chapter 4.

You can also use dash instead (I usually do) like [13-15] or leave off the leading/trailing numbers to implicitly mean 'from the beginning' and 'to the end' eg: [-5] or [5-].

Examples:
https://www.fanfiction.net/s/11491965/1/[-4]
https://www.fanfiction.net/s/11491965/1/[17]
https://www.fanfiction.net/s/11491965/1/[48-]

By default, the title of the story will be modified to include the chapter range for most sites, eg Memories Of Others (Ch 1-4). But not on XenForo sites.

[readerboy7]

Ah, thanks for informing me.
By the way; sorry if this is off topic, but is the story you linked to a good one? The first chapter seems interesting, although some aspects are kinda worrying.

[JimmXinu]

Originally Posted by readerboy7

By the way; sorry if this is off topic, but is the story you linked to a good one? The first chapter seems interesting, although some aspects are kinda worrying.

TheBeardedOne has written a number of stories I've found fairly enjoyable. Generally feel good, non-angsty, good guys win type stories with some mild romance. I don't remember anything NSFW in any of them it that's what's worrying you.

Please reply by PM if you'd like to discuss more.

[Feynor]

Originally Posted by JimmXinu

replace_br_with_p is a heuristic or 'best effort' feature. It's never going to be perfect. And I believe it was designed to replace double <br> tags between paragraphs with <p> tags, not to unwrap lines.

The problem is with the way ffn and others does things, they really screw up everything. You post a story, and what does ffn do? they replace each paragraph with br-br, destroying important structural information.

Sadly sometime FFn or an author have no discernible structure, with parts of the chapter being vastly different from the rest, and in those cases replace_br_with_p throws its marbles and either does nothing (if you are lucky), or goes on a rampage on the chapter.

replace_br_with_p tries to determine what number of breaks should count as a new paragraph. Anything less than that is supposed to be collapsed, more is seen as segment breaks.

You can't just say "two breaks or more = paragraph", as some have that many breaks per line, and for instance four breaks paragraphs.
This is often the case for older stories, where the author may have imported an email formatted story into ffn, or word, before importing THAT into ffn. ffn then replaces each line break with br-br.

Just look at this text. Because mobileread doesn't have proper formatting for paragraphs, I'm using double line breaks to emulate those paragraphs. In ffn that would translate to 4 br tags.

[edeniz]

Hi! I've stumbled on a few confusing problems:

1) "Overwrite If Newer" does not seem to be working anymore when the format in question is "html". At least not on ffnet and ao3 both. The error message is,

Not Overwriting, web site is not newer.

And yet the story in question has updated. Not just that, but the solution seems to be to either simply use, "Overwrite Always" or just "Add New Book".

I don't know how long this has been going on (I only noticed it today), or if other archives are affected as well. And though I did look through the past several pages of the thread, I haven't seen anyone else report anything similar either. Is it really just me?

If necessary, I could attach an example fic each for ffnet and ao3 or the debug log.

2) tth seems to be on the fritz yet again. It is impossible to download from there anymore. I get the prompt to log in with username and password, and even after entering them, the prompt window keeps popping up again and again until I give up and press cancel. Is this a known thing? A mess-up on tthfanfic.org's part that we can only wait out? (Why oh why is it always tth? *sigh*)

And to clarify: I'm using the latest plugin version, but not the test version.


EDIT:
I forgot to mention something else as well, so better add it here instead of making another post, right?

exclude_metadata_post is supposed to be executed after replace_metadata, right? Well. It doesn't seem to work that way for me.

Example link:

http://archiveofourown.org/works/246947[1]
The Jenny Code (Ch 1-1)
Authors: Domenika Marzione (domarzione)

(rather long fic, so it's better download only one or two chapters.)

I tried to move "Genii" from genre to characters. Doing this works, in that "Genii" is copied to "characters":

characters=>(.*)=>\1\,Genii&&genre=>Genii

But when I then try to get rid of the genre "Genii" after the replacement via exclude_metadata_post, the downloaded work will not have "Genii" anywhere, be it genre or characters.

And no, I checked, I did not put "Genii" accidentally inside exclude_metadata_pre or some such thing.

Again, no idea if it is something new or if it's been this way for a while. It's not that it's supposed to be this way and I simply misunderstood what exclude_metadata_post is supposed to do, right?

[JimmXinu]

Originally Posted by Feynor

The problem is with the way ffn and others does things, they really screw up everything. You post a story, and what does ffn do? they replace each paragraph with br-br, destroying important structural information.

Absolutely. I wasn't trying to criticize replace_br_with_p, just trying to explain the limitations as I understand them.

(FYI, Feynor is the one who gave us the feature replace_br_with_p, so he's the expert on it.)