[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by Tanjamuse

I have URL columns for Author and Story (not the same column), it's comments-columns.

Is it possible to make the URL's clickable instead of just plain-text?

You can fill a column with the authorHTML instead of just authorURL with this:

custom_columns_settings:
 authorHTML=>#author_url

It does not become a clickable link for me in Calibre column. You'll need tot ask in the Library Management forum about that.

(Didn't we already have this conversation before?)

[Tanjamuse]

I tried searching, but couldn't find my question for earlier. Sorry for the repeat question.

[version365]

Originally Posted by davidfor

The other thing I noticed while doing this is that a lot of the styles don't show as the default used by FFF doesn't include them. I have grabbed the stylesheet from the site and cut out all the styles that don't make it to the epub. I'm not sure whether to include this as the default styles for the site in the plugin. If you want to try and see if it does improve the downloaded books, here it is. Just add all of into the personal.ini.

Spoiler:

[storiesonline.net:epub]
output_css: 
 *{box-sizing:border-box}
 h1,h2,h3{text-align:center;line-height:120%%}
 h1{font-size:2em}
 h2{font-size:1.6em}
 h3{font-size:1.4em}
 h4{font-size:1.15em}
 h5{font-size:1.1em}
 h6{font-size:1.05em}
 small{font-size:0.93em}
 big{font-size:1.07em}
 td{text-align:left;vertical-align:top;padding:2px}
 th{text-align:center;vertical-align:top;padding:2px}
 .cen{margin-left:auto;margin-right:auto}
 .thick{height:0;border-top:#666 6px solid}
 blockquote{margin-left:3em;margin-right:3em;border-left:2px #ddd solid;padding-left:8pt}
 blockquote p{font-family:Courier, monospace}
 .right,.r{text-align:right}
 .just,.j{text-align:justify;text-indent:2em}
 .in{text-indent:2em}
 .center,.c{text-align:center}
 .red{color:red}
 .blue{color:navy}
 .lblue{color:blue}
 .green{color:green}
 .i{font-style:italic}
 .b{font-weight:bold}
 .bi{font-weight:bold;font-style:italic}
 .lb{background-color:#9DF}
 .lg{background-color:#9FB}
 .imgsp{border:1px maroon solid;margin:10px}
 .notice,.end-note{background-color:#FFE;padding:5px;border:1px dotted silver;margin-bottom:5px;color:navy}
 .notice p,.end-note p{font-family:Helvetica,sans-serif;color:navy}
 .notice p{font-size:90%%}
 .invis{display:none}
 .ind-e{margin-top:10px;margin-bottom:5px;font-family:sans-serif;font-weight:bold}
 .pic-left{float:left;margin:5px}
 .pic-right{float:right;margin:5px}
 .pic-center{margin-left:auto;margin-right:auto;clear:both;display:block}
 .ind-note{font-size:0.85em;font-family:sans-serif;margin:0 40pt 6pt 35pt;font-style:italic}
 .str{text-decoration:line-through}
 .tt{font-family:Courier, Monaco, monospaced}
 hr{width:30%%}
 .fr{float:right}
 small{font-family:Helvetica,sans-serif;font-size:9pt}
 td{text-align:left}
 blockquote{margin-left:3em;margin-right:3em;font-family:Courier, monospace}
 .right,.r{text-align:right}
 .just,.j{text-align:justify;text-indent:2em}
 .center,.c{text-align:center}.in{text-indent:2em}.red{color:red}.blue{color:navy}.lblue{color:blue}.green{color:green}A:link{color:maroon}A:visited{color:red}
 p{text-align:left;font-family:Georgia,serif;color:#000;word-break:break-word}
 .i{font-style:italic}.b{font-weight:bold}
 .bi{font-weight:bold;font-style:italic}
 .fr{float:right}#story{padding:2em}}
 img{max-width:100%%;height:auto}

Originally Posted by davidfor

There are probably only a few more that are important. And, like you, I'm hoping some of the other readers put in an opinion.

This is really a good idea.. It would give the feel of the specific site.. also a little variation.. which I personally like..

Originally Posted by davidfor

The other is a "user site stylesheet" option.

This is great too!

Originally Posted by JimmXinu

Pulling stylesheets from the sites seems prohibitively complex. A page can include many stylesheets (including common ones like jquery-ui) as well as on-page styles. And each site can use some different combination. And then there's different site themes/skins which already causes problems sometimes.

I think the solution in this case should be that, first FFF gets the raw text as it is now, and then the text would be put in a user-created site-specific stylesheet.
So it's upto the user how he/she wants to design that stylesheet, and also FFF doesn't have to worry about stylesheets on the webpage.

[JimmXinu]

Originally Posted by version365

I think the solution in this case should be that, first FFF gets the raw text as it is now, and then the text would be put in a user-created site-specific stylesheet.
So it's upto the user how he/she wants to design that stylesheet, and also FFF doesn't have to worry about stylesheets on the webpage.

So that's basically unchanged from today? Except for possibly adding an option to pull the CSS from an external file/URL instead of keeping them in personal.ini?

[davidfor]

Originally Posted by JimmXinu

So that's basically unchanged from today? Except for possibly adding an option to pull the CSS from an external file/URL instead of keeping them in personal.ini?

Yes, I think so. Being able to use a URL for the included stylesheet would solve my "use site stylesheet" suggestion. And if the adapter creators find a suitable URL for the stylesheet, then this can be included in configuration files but commented out so users can use it if they want to.

You probably don't want to open this..

Spoiler:

Of course, allow multiple includes so that I can use the site stylesheet and override with mine. Just add them to the epub and link them in the order listed.

Easy, right?

[JimmXinu]

Originally Posted by davidfor

Yes, I think so. Being able to use a URL for the included stylesheet would solve my "use site stylesheet" suggestion. And if the adapter creators find a suitable URL for the stylesheet, then this can be included in configuration files but commented out so users can use it if they want to.

That seems fair. Another possible option would be adapter creators putting FFF-specific-site-specific CSS files in github or somewhere and linking to those.

Originally Posted by davidfor

You probably don't want to open this..

Spoiler:

Of course, allow multiple includes so that I can use the site stylesheet and override with mine. Just add them to the epub and link them in the order listed.

Easy, right?

Listing more than one file/URL in INI is a possibility--and I'd much rather have the suggest before I add code rather than after.

But if I do that, they'll just get concatenated together into one CSS file in the epub; which should be functionally equivalent. And for html output, the CSS gets embedded.

There are some complex interactions I want to consider first about precedence and ordering for files, CSS in the INI, and [sections] order.

I'm also considering the possibility of making it a general feature (like add_to_), applicable to other (all?) settings. replace_metadata comes to mind.

[davidfor]

Originally Posted by JimmXinu

That seems fair. Another possible option would be adapter creators putting FFF-specific-site-specific CSS files in github or somewhere and linking to those.

I was sort of thinking of this, but hadn't come up with a location.

Listing more than one file/URL in INI is a possibility--and I'd much rather have the suggest before I add code rather than after.

I know exactly what you mean. I'm about to turn to someone in my team and tell them that when they finish the code they are working on and put it into production next week, they are to start working on rewriting it because the @%#^&@* business people forgot to mention all the exceptions to the rules.

But if I do that, they'll just get concatenated together into one CSS file in the epub; which should be functionally equivalent. And for html output, the CSS gets embedded.

Yes, that's probably simpler

There are some complex interactions I want to consider first about precedence and ordering for files, CSS in the INI, and [sections] order.

I'm also considering the possibility of making it a general feature (like add_to_), applicable to other (all?) settings. replace_metadata comes to mind.

Good idea. Or at least make it reusable so that when you do find other uses, you can.

[mehetabelo]

Alright, so I want to point out that this was my mistake and it almost caused me to cry, but it brought up a possible issue.

Skip the next few paragraphs if you don't feel like reading my stupid story. I'll identify the issue below clearly.

I used to keep multiple libraries, the reason is that my HDD was slow and large libraries made the response time unreasonable for me. I kept some major libraries separate from years ago. Recently after purchasing a large SSD just for calibre library use I started merging everything into a couple of separate libraries, keeping my FFF library separate due to it having a lot of extra custom columns and settings.

I have done a lot of customization on my default ini to the point where it's ~600 lines long (with a blank line between each site). I do occasionally copy it out and keep a text backup and I did one in January. The problem came yesterday. I opened my FFF library and went to download an updated book... and problem... the default ini had somehow got reset during the move to the SSD. I'm not entirely sure how that happened since I literally just copied the entire calibre folder over, but it happened. The original file folder had just been deleted, since I'd confirmed that everything had moved and the file sizes matched up. I panicked. I went through my backup and found it was almost 2 months old and I'd done a LOT of changes since then, including adding every Wuxiaworld story in to manually edit the story title and author. I checked some of my other libraries to see if my default.ini in those were similar to the more recent since I'd done a couple of 'new' libraries in the merging process. Those had been copied from a different library that didn't have all the columns for FFF, and thus didn't have the same default.ini copied over.

After about 10 min I discovered that I wasn't going to go on a murdering rampage. I'd created a new library for a friend, trying to get him into using FFF instead of manual downloads. He's a lazy ass, but he's so backwards that he's too lazy to use FFF since he's used to using another method and doesn't want to 'learn' how to use FFF. No matter how much I explain it'll go faster and easier... anyway. I'd created a new library for him and the defaults.ini was a part of it. I'd edited my usernames/passwords out, but it had everything else so a lot of it was setup for him.

Unfortunately the idiot still won't use Calibre. *sigh*. BUT my wasted time wasn't so wasted apparently.

With a few minutes and some matching of username/passwords I had my defaults.ini almost back to the place where it was before. I had to add some edits I'd made just a couple days previously but that was easy since they were similar for each site and I could remember exactly what I'd done. (I added a new column for virtual libraries and made it a bit easier to sort, so I added custom metadata to sort into a library based on type, fanfiction, webfiction...). The only other changes I had to do was fix some of the other options I'd made, choosing the settings I'd been using previously, output options, gui options.

I did notice that despite almost everything resetting my email info and my Custom Columns tab did not reset. So, whatever happened had to be my fault I'm sure, but some settings did reset like everything in the 'Basic' tab. 'Default to update books when selected' was checked, and I have had that unchecked for years. It was actually how I noticed the problem since I went to paste the url in and it tried to update a non-fff book that I had highlighted.

The point of this all:

Anyway, the point of this long and overly specific tale is that I was thinking having a backup option isn't a bad idea. Rather than having to copy/paste my defaults.ini into a text file as a backup option, is there an easy way to implement a click to backup option? This would ideally include the other options that have been made like the Custom Columns tab, emails settings, basic options, etc. If my database wasn't so big I'd just keep that as a backup but it's literally 336MB at the moment and backing that up online every time my library changes is… not really an option for me.

Second, a possible Issue/bug

Again, this is again something that is my fault but I noticed some (probably) incorrect behavior because of it. I may be reading the entire thing incorrectly but I thought I'd point it out and let you decide.

When I brought up Wuxiaworld stuff a few weeks back I went through and setup all author/titles. I did a lot of work in the columns including setting up the Genre (separate from tags) for each book using NovelUpdate information. Originally I had a metadata catch to see if the plugin grabbed the metadata. Custom Column setting was: " genre=>#genre,a" There apparently was no Genre, so I added everything manually and went on about my business. I noticed today that when I updated a book that that setting wiped out the Genre column. With the ",a" at the end of that, should it not 'add' onto the column only? If I remember right there are 3 options, a (add) r (replace) and n (for newly downloaded stories). So, even though 'genre' isn't picking up anything, why would it be wiping out my custom column?

It's annoying, but I can put the genre's back in without too much work, though it will take some time. It's not a big deal, I just took the setting out of my custom column setting. But as I said, based on the action selected (a) should this not have been left alone in the first place?

[JimmXinu]

Originally Posted by mehetabelo

...

Ouch. I'm glad you were able to recover.

I did some investigating just to make sure: Copying a calibre library directory to another calibre instance on an other machine works as I expected, carrying the FFF config with it.

(FYI, I did find that copying a library dir to a new name within the same calibre instance/install doesn't work as I expected. Calibre kinda-sorta considers them the same--same library UID and shared FFF settings. So don't do that.)

In other news, be careful with Calibre's export/import all data features. It means it when it says 'all'.

Originally Posted by mehetabelo

...is there an easy way to implement a click to backup option? This would ideally include the other options that have been made like the Custom Columns tab, emails settings, basic options, etc. If my database wasn't so big I'd just keep that as a backup but it's literally 336MB at the moment and backing that up online every time my library changes is… not really an option for me.

As for backup, I submit that asking each program (let alone each plugin in each program) to provide their own backup mechanism is not practical.

I think what you need is incremental backup of the library dir, not manual backup of specific parts. My library is 2.25G and my incremental backup (rsync) generally takes only a few minutes because it's only sending what's changed.

Originally Posted by mehetabelo

...
Custom Column setting was: " genre=>#genre,a" There apparently was no Genre, so I added everything manually and went on about my business. I noticed today that when I updated a book that that setting wiped out the Genre column. With the ",a" at the end of that, should it not 'add' onto the column only? If I remember right there are 3 options, a (add) r (replace) and n (for newly downloaded stories). So, even though 'genre' isn't picking up anything, why would it be wiping out my custom column?

I just tested that and in personal.ini, custom_columns 'a' worked as expected for me.

However, if you use the setting on the Custom Columns tab to set the column, it always does replace. Even if you have 'a' in custom_columns. Might that explain it?

[skb]

Well, after all these years, this is just embarrassing...where is the personal.ini (I did a search on my hard drive...not there) and assuming I need to create one, where does it live once created? Is there a default version to copy?

Edit: Oh well, that's just doubly embarrassing. It's in the FFF config in Calibre. Doh! Of course it is.

[JimmXinu]

Originally Posted by skb

Well, after all these years, this is just embarrassing...where is the personal.ini (I did a search on my hard drive...not there) and assuming I need to create one, where does it live once created? Is there a default version to copy?

It's called personal.ini because it was a file named that in the original CLI version. (Still is, if you use the pip installed CLI.)

In the Calibre plugin version, it's editable from FanFicFare configuration.

[skb]

Thank you, Jim. You are a prince among men.

I got it working beautifully despite my denseness.

[JimmXinu]

While testing some changes, I've found a bunch of supported sites have disappeared.

If you have reason to believe that any of these will be back soon, please speak up. Otherwise they are doing to be removed from FanFicFare.

• www.restrictedsection.org - Redirects to Harry Potter LEGO on Amazon.com.
• harem.lucifael.com - Account suspended
• fanfiction.lucifael.com - Account suspended
• www.phoenixsong.net - 504 Gateway Time-out
• onedirectionfanfiction.com - Internal Server Error
• samdean.archive.nu - Redirects to 127.0.0.1
• www.hpfandom.net - No Response
• www.ficsite.com - No Response
• www.sinful-desire.org - redirects to AO3
• thehexfiles.net - redirects to AO3

[version365]

@JimmXinu,
In the latest test version you removed portkey.org and psychfic.com adapters, but didn't remove these sites from default.ini
So, every time I try to save personal.ini, it shows error "Bad Section Name"

[JimmXinu]

Originally Posted by version365

@JimmXinu,
In the latest test version you removed portkey.org and psychfic.com adapters, but didn't remove these sites from default.ini
So, every time I try to save personal.ini, it shows error "Bad Section Name"

Yep. That's why we test. I found that, just didn't release a new test version for it.

There are another 10 sites that are probably going to be removed soon, too.