[GUI Plugin] FanFicFare

[texasnightowl]

Turns out I do have a specific question. I mentioned above I created the custom column to save metadata which is working fine. But...some of that data I don't really care about. Like the collections data.

I'm confused about how to exclude that.

If I don't want that collections data pulled at all...I want it either omitted or deleted from the saved metadata column, how do I exclude it or remove it?

Does just removing the collections entries from extra_valid_entries do that? Or do I need to do an exclude_metadata_pre/post of some sort?

[MidnightRose]

Originally Posted by JimmXinu

NOTE: I make no promises that any particular Kindle or other device can render a CSS background image. I'll honestly be more surprised if it can.

Did some testing! So it turns out that if you edit a book in the calibre epub editor, there's a menu option to download any remote assets, which does all those steps in one go.

And yes, it turns out my kindle does show the background image just fine.... aside from resolution issues. Said resolution issues make this particular instance completely unreadable, but hat's just how things go.

I've included a screenshot.

[JimmXinu]

Originally Posted by texasnightowl

If I don't want that collections data pulled at all...I want it either omitted or deleted from the saved metadata column, how do I exclude it or remove it?

From FFF's 'Saved Metadata column'? You can't control that, by design.

It contains everything FFF collected without any of the replacement, in/exclude processing so when you change your mind about what you want in your own custom columns, it's all still there.

You can tweak which columns Calibre searches by default in Calibre's prefs > Search if you want it to be more invisible.

[JimmXinu]

Originally Posted by MidnightRose

Did some testing! So it turns out that if you edit a book in the calibre epub editor, there's a menu option to download any remote assets, which does all those steps in one go.

I did look for that in Edit--it's down a Tools > External links submenu and I was looking for 'Download...'

FYI, that does the same thing as Polish book--it works, but next the FFF update will break it.

Originally Posted by JimmXinu

...
Calibre's Polish book has an option to "Download external resources" that works to get remote CSS images in the epub. But they will be discarded on the next FFF update. Fine for completed stories, less good for actively updating stories.


You know, I just remembered I added code to EpubSplit to find images in CSS a while ago. I wonder if I could make that work in FFF...

I'll look into it tomorrow.

One of the key parts of FFF's image processing is storing the original image URL to match duplicates in different chapters. We store that in the <img longdesc="origurl"...> attribute.

CSS url() includes don't have anything like that we could appropriate. Trying to store origurl in CSS comments or something seems both difficult and sketchy... We do have adedup_img_files setting now, but that doesn't save image downloads, just space in epub.

I know this has come up a couple times before (in 15 years), but how often are CSS images actually a problem?

[texasnightowl]

Originally Posted by JimmXinu

From FFF's 'Saved Metadata column'? You can't control that, by design.

It contains everything FFF collected without any of the replacement, in/exclude processing so when you change your mind about what you want in your own custom columns, it's all still there.

You can tweak which columns Calibre searches by default in Calibre's prefs > Search if you want it to be more invisible.

ah, ok. I did find the setting to remove it from the book details section of the main display, so I guess I'm good.

and I managed to stumble my way to a replace command I'm contemplating. (Trying to decide if I want to just simply the Marvel fandoms and change all of them (Iron Man, Captain America, Thor, Avengers) to just Marvel or MCU. All the characters end up showing up in all the fanfics basically so...now just playing around with things.)

At some point I'll have to start checking some of the other sites I still visit, but right now my "new" personal.ini is only for AO3 since that is where I do the majority these days.

[Eridyne]

I just got started playing around with FFF and my new Libra Colour, and I'd really like to basically fully automate putting fanfics onto my e-reader.

Seeing as I mostly read SpaceBattles and SufficientVelocity, and watch all the threads I intend to follow, is there a way to grab the complete watchlist for a xenforo account, even if multi-page, and use that as a URL list for download/update? If there is, I can't figure out how.

[whiteowl]

Question about epub/image_max_size option.

Is there a way to disable resizing at all? Or should I just set that option to some huge value so that squeezing image would not happen?
What huge numbers for image_max_size have sense?

[JimmXinu]

Multireply

Originally Posted by Eridyne

I just got started playing around with FFF and my new Libra Colour, and I'd really like to basically fully automate putting fanfics onto my e-reader.

Seeing as I mostly read SpaceBattles and SufficientVelocity, and watch all the threads I intend to follow, is there a way to grab the complete watchlist for a xenforo account, even if multi-page, and use that as a URL list for download/update? If there is, I can't figure out how.

Not specifically.

There is a relatively generic 'Get Story URLs from Web Page' feature, but it doesn't know how to navigate paged lists on most sites. And it doesn't know to attempt to login on XF sites (SB/SV/etc).

I was going to tell you about how to use copy/paste from watched threads in browser, into FFF's Story URLs dialog box; but that gives the threads and the last post, which is also a valid story URL. So you can use it, but you'll have to edit the list.

Another issue is that I do not recommend downloading more than maybe 12 stories at a time from any site. 'Excessive' requests is a good way to get blocked, but 'excessive' varies.

Originally Posted by whiteowl

Question about epub/image_max_size option.

Is there a way to disable resizing at all? Or should I just set that option to some huge value so that squeezing image would not happen?
What huge numbers for image_max_size have sense?

You can disable all image processing with:

no_image_processing:true

Other than that, yes, just use huge numbers.

[Eridyne]

I'd seen Get Story URLs from Web Page, and discovered its inability to handle logins already, unfortunately. I'd assumed it probably couldn't handle pagination, but was hoping it could at least do logins.

I'm aiming for as much automation as possible, so copy paste from watched threads plus manual edits isn't the best, but would be better than nothing.

Is there not a way to throttle downloading sufficiently, in your opinion, even in the case of very large URL lists?

[JimmXinu]

Originally Posted by Eridyne

Is there not a way to throttle downloading sufficiently, in your opinion, even in the case of very large URL lists?

Basically, no.

Most of the big sites we support now use Cloudflare or some equivalent that throttles or blocks automated traffic when it becomes 'excessive'. Sometimes people get blocked entirely for days. Or they leave it maximum blocking mode forever, like ffnet.

How much is excessive? Depends on the site. I choose not to try to find the lines -- I still want to be able to download for my own use.

You can set longer slow_down_sleep_time values for different sites and experiment if you want.

I will bump up the default slow_down_sleep_time for a site when users report that increasing it helps.

[Demanih]

Is it possible to add a fanfiction site to FFF "ficbook.net" they recently disabled the ability to save fanfiction without a paid subscription. (((

[JimmXinu]

Originally Posted by Demanih

Is it possible to add a fanfiction site to FFF "ficbook.net" they recently disabled the ability to save fanfiction without a paid subscription. (((

ficbook.net is already supported.

However, due to cloudflare blocking, I have to use BrowserCacheFeature to download from it.

[MidnightRose]

Is the plugin able to set a chapter range to download, or only the commandline version?

reason I ask is that some of these webnovels get unreasonably long, so removing everything before the current story arc would be nice both in terms of file conversion time, as well as having reasonable values in the "Percentage of the way through the book" indicator.

(A related nice thing to have would be some way to split apart long series into separate books, but that seems more annoying to do if the original isn't already split in an easy to differentiate automatically way. Looking at you, royalroad.)

[JimmXinu]

You can:

• Put a chapter range on end of a story URL, example: https://www.royalroad.com/fiction/104434/[45-65] You can also do [5] (single chapter 5) [-5] (all before and chapter 5) and [5-] (chapter 5 and all after). The down side is it isn't remembered if you want to update the book.
• Use the 'undocumented' test/dev first_chapter and/or last_chapter in personal.ini in that story's [section]. This is equivalent to the [x-y] range, but saved so it is applied on updates.
• Use EpubSplit plugin (see my sig for link) to split out selected chapters of a downloaded epub.

FYI, I'm working on url() refs in CSS, but it substantially changes how FFF saves images. I'm not even using it myself yet.

[ownedbycats]

Running checks in epub-editor, noticed FFF downloads have Unexpected unknown property "adobe-hyphenate" (property-no-unknown). Supposed to do something?