[GUI Plugin] FanFictionDownLoader - Page 64

kovidgoyal · 10-18-2012, 10:52 PM

Quote:

Originally Posted by davidfor

It does, but it does it differently to the conversion process. Don't ask me how it is different, but it has been discussed on the forum somewhere.

The viewer runs only the input plugin part of the conversion, which converts the file to an OEB book (html+opf files).

Jade Aislin · 10-20-2012, 01:04 PM

I've been excited about the different changes that have been made to this plugin, especially the replace metadata and the ability to include the extra metadata that some sites have. I'm finally at the point that I am ready to start playing with these features.

First I'm trying to use the replace metadata feature to fix a problem that I have had since I started using FFDL. Two different characters from different fandoms use the same name in FFN, Supernatural's Sam W. (Sam Winchester) and Transformers' Sam W. (Sam Witwicky). I was hoping to use the replace metadata so that I don't have to do a manual replace everytime I downloaded a story with either of those characters.

However, I cannot seem to get the replace feature to work.

Some of the different code I tried using include (although it is not shown, there is a space at the beginning of the last line of code for each example):

Spoiler:

I am not sure what I am doing wrong, but I cannot get Sam W. to change to Sam Witwicky.

- Jade Aislin

JimmXinu · 10-20-2012, 03:14 PM

Quote:

Originally Posted by Jade Aislin

First I'm trying to use the replace metadata feature to fix a problem that I have had since I started using FFDL. Two different characters from different fandoms use the same name in FFN, Supernatural's Sam W. (Sam Winchester) and Transformers' Sam W. (Sam Witwicky). I was hoping to use the replace metadata so that I don't have to do a manual replace everytime I downloaded a story with either of those characters.

Hmmm. That's not an easy problem. In fact, it can't be done with the existing replace_metadata feature.

Your first problem in all cases is that the section name needs to be [www.fanfiction.net], not [http://www.fanfiction.net/]. With that fixed, both of your latter two examples work.

But even with that, it needs a new feature to make it work. And it's interesting enough I've taken a stab at it.

With the attached version, try this:

Code:

[www.fanfiction.net]

## *Five* part lines.  Effect only when trailing conditional key=>regexp matches
## metakey[,metakey]=>pattern=>replacement[&&metakey=>regexp]
replace_metadata:
 characters=>Sam W\.=>Sam Witwicky&&category=>Transformers
 characters=>Sam W\.=>Sam Winchester&&category=>Supernatural

It's similar to your intuitive first try above, but puts the conditional at the end with a different separator.

This will fail in the face of a Transformers/Supernatural crossover, but I don't see any way around that.

Jade Aislin · 10-20-2012, 04:34 PM

Quote:

Originally Posted by JimmXinu

Your first problem in all cases is that the section name needs to be [www.fanfiction.net], not [http://www.fanfiction.net/]. With that fixed, both of your latter two examples work.

I thought I had missed something when I put the code in. I didn't even realize that I had the 'http' on it.

Quote:

Originally Posted by JimmXinu

But even with that, it needs a new feature to make it work. And it's interesting enough I've taken a stab at it.

With the attached version, try this:

Code:

[www.fanfiction.net]
 
## *Five* part lines.  Effect only when trailing conditional key=>regexp matches
## metakey[,metakey]=>pattern=>replacement[&&metakey=>regexp]
replace_metadata:
 characters=>Sam W\.=>Sam Witwicky&&category=>Transformers
 characters=>Sam W\.=>Sam Winchester&&category=>Supernatural

It's similar to your intuitive first try above, but puts the conditional at the end with a different separator.

This will fail in the face of a Transformers/Supernatural crossover, but I don't see any way around that.

Thanks. It worked great. I know that it wouldn't be able to figure out which name to use if there was a crossover, but how would it respond if that happened? Would it randomly choose one or just leave it as Sam W.? Either way, I'll probably just have to do any story like that manually. I don't mind doing a few. Although, I don't think that there is any Transformer/Supernatural crossover that I want that has the character Sam W. so it may not be a problem for me.

JimmXinu · 10-20-2012, 05:16 PM

Each replace_metadata line is checked in order for regexp and conditional match. So the first one will be applied.

However, if the changed value matches a later line, that later line will also be applied.

A completely made up example:

Code:

replace_metadata:
 genre=>Sci-Fi=>SF
 SF=>San Fransisco

Sci-Fi would be converted first to SF, then SF to San Fransisco. Reversing the order of the two lines would prevent that in this case.

crackshot91 · 10-22-2012, 02:46 PM

Hey Jimm. Another question for ya.

Is there a way to have a column that displays the rating of a fic from fimfiction? Not the content rating, but the thumbs up/thumbs down rating. I'm not sure how to go about creating the column or even if FFDL downloads the quality rating as metadata.

Thanks!

JimmXinu · 10-22-2012, 03:06 PM

@crackshot91 - I can add a new site specific metadata for it if you can give some example URLs. I don't read that site, so I don't know what they usually look like.

Ideally, I'd like to see a story with each rating and one without a rating (if that can happen).

crackshot91 · 10-22-2012, 03:29 PM

Quote:

Originally Posted by JimmXinu

@crackshot91 - I can add a new site specific metadata for it if you can give some example URLs. I don't read that site, so I don't know what they usually look like.

Ideally, I'd like to see a story with each rating and one without a rating (if that can happen).

JimmXinu · 10-22-2012, 07:13 PM

New test version with additional metadata for www.fimfiction.net.

New entries are: likes,dislikes,views,total_views,short_description

Added to defaults.ini (for reference, you don't need to add these to personal.ini unless you want to change them.:

Spoiler:

To add these to the titlepage inside the ebook, use extra_titlepage_entries or titlepage_entries.

To set these to calibre custom columns, use custom_columns_settings.

This version also adds a check to prevent infinite recursion if a replace_metadata conditional refers to the same entry it's trying to change.

crackshot91 · 10-22-2012, 07:51 PM

EDIT: OH GOD I'M NOT VERY SMART.

I had "custom_column_settings" instead of "custom_columns_settings" in the override section of the ini. Please ignore my stupidity.

ANOTHER EDIT: I was wondering if there was a way to get a custom column to report the ratio of likes:dislikes? What I'm working toward is being able to sort first by total views, and then by likes/Dislikes ratio to approximate fimfiction's rating system (And generally just browse my catalog easily by rating.) What you've done so far is wonderful, so I don't want to push it.

ANOTHER EDIT X2: Gah. I feel like I'm being a nuisance. I've added "dislikes=>#dislikes,r total_views=>#views,r" to the "custom_solumns_settings" in the personal ini. Now I get an error. "too many values to unpack". The custom columns entries only need to be separated by a space, right? Is it possible to have more than one custom column entry in the personal.ini?

Everything below this line is silliness:
-------------------------------------------------------------------------------------

Thank you for your help, Jimm! Unfortunately, I feel kinda stupid right now.

I'm looking at this page of the thread. Specifically this part:

Code:

## (Plugin Only) - You can also populate calibre custom columns with
## the site specific metadata using custom_columns_settings (but only
## if 'Allow custom_columns_settings from personal.ini' is checked in
## the plugin GUI config.) There are three parts, the entry name,
## then the label of the calibre custom column, then (optionally) a
## 'mode'. 'r' to Replace any existing values, 'a' to Add to existing
## value (use with tag-like columns), and 'n' for setting on New books
## only.
## Make sure to keep at least one space at the start of each
## line.
custom_columns_settings: cliches=>#acolumn,r themes=>#bcolumn,a timeline=>#ccolumn,n

But I can't seem to get the custom columns to work right. If I understand correctly, if I wanna have a column for likes, I need:

custom_columns_settings: likes=>#Likes,r

Is that right? What is meant by the "Label of the calibre custom column"? Is that the lookup name? Or the Column Heading? Because I've tried both. The lookup name for the likes column is "likes" and the column heading is "Likes".

I don't think I'm doing this right.

Just to check: the likes column needs to be an integer column, right? Format: {0:}

Also, I have FFDL set to allow custom_column_settings to overwrite.

JimmXinu · 10-22-2012, 08:44 PM

Quote:

Originally Posted by crackshot91

EDIT: OH GOD I'M NOT VERY SMART.

I had "custom_column_settings" instead of "custom_columns_settings" in the override section of the ini. Please ignore my stupidity.

Okay.

Quote:

Originally Posted by crackshot91

ANOTHER EDIT: I was wondering if there was a way to get a custom column to report the ratio of likes:dislikes? What I'm working toward is being able to sort first by total views, and then by likes/Dislikes ratio to approximate fimfiction's rating system (And generally just browse my catalog easily by rating.) What you've done so far is wonderful, so I don't want to push it.

Actually, I did think about that.

I considered making one field of likes/dislikes instead of two fields, but then it would be a string, not a number. 9/100 would sort 'higher' than 200/0.

You could create a composite custom column in calibre that displays likes/dislikes but it would have similar ordering issues.

One way would be to use View Manager plugin to order by total views(desc), likes(desc), dislikes(asc) while displaying views and likes/dislikes. But lots of custom columns (especially composite custom column) slows calibre significantly when you get past ~1000 books. I used to use several composite custom columns, but I've eliminated most of them from my main library for speed's sake.

Quote:

Originally Posted by crackshot91

ANOTHER EDIT X2: Gah. I feel like I'm being a nuisance. I've added "dislikes=>#dislikes,r total_views=>#views,r" to the "custom_solumns_settings" in the personal ini. Now I get an error. "too many values to unpack". The custom columns entries only need to be separated by a space, right? Is it possible to have more than one custom column entry in the personal.ini?

They have to be on separate lines, and each line has to start with one space.

Code:

custom_columns_settings:
 likes=>#Likes
 dislikes=>#dislikes
 total_views=>#views

As a general note: the forum only notifies subscribers when a reply is posted, not when an edit to a post is made. You got lucky this time--I didn't look until after the first two edits and the third happened between when I read it and clicked 'quote'--but it's not uncommon for me to hit 'quote' and spent an hour composing my reply what with interruptions and stuff.

So, it's generally better, IMO, to add more posts than to edit an existing one. FYI.

crackshot91 · 10-22-2012, 08:53 PM

Oh god thank you.

I put the custom columns settings entries on separate lines. Works great now.

Also, composite columns seems like a good way to go. I'll give that a try.

And thanks for the note about the lack of edit notifications. I had a feeling that was the case, but I really didn't want to spam the thread. I see now that it may have been necessary. Imagine if you saw my post before the edits! D:

Thanks again for your help! You rock!

JimmXinu · 10-23-2012, 05:03 PM

Version 1.6.15 - 23 Oct 2012

Fix for some stories on hpfandom.net
Fix for some stories on fimfiction.net
Add conditional replace_metadata feature
Extra metadata for fimfiction

thecookiemommma · 10-24-2012, 03:12 PM

Thank you, Jimi for all your hard work. I use this all the time, and it seems to be pretty darn awesome. It's made collecting fic to read nearly seamless.

Thank you!

JimmXinu · 10-24-2012, 03:56 PM

Quote:

Originally Posted by thecookiemommma

Thank you, Jimi for all your hard work. I use this all the time, and it seems to be pretty darn awesome. It's made collecting fic to read nearly seamless.

Thank you!

10-20-2012, 01:04 PM	#947
Jade Aislin Groupie Posts: 164 Karma: 3100 Join Date: Sep 2011 Device: Kobo Auro H2O, PRS-T1	I've been excited about the different changes that have been made to this plugin, especially the replace metadata and the ability to include the extra metadata that some sites have. I'm finally at the point that I am ready to start playing with these features. First I'm trying to use the replace metadata feature to fix a problem that I have had since I started using FFDL. Two different characters from different fandoms use the same name in FFN, Supernatural's Sam W. (Sam Winchester) and Transformers' Sam W. (Sam Witwicky). I was hoping to use the replace metadata so that I don't have to do a manual replace everytime I downloaded a story with either of those characters. However, I cannot seem to get the replace feature to work. Some of the different code I tried using include (although it is not shown, there is a space at the beginning of the last line of code for each example): Spoiler: [http://www.fanfiction.net/] replace_metadata: category=>Transformers=>characters=>Sam W\.=>Sam Witwicky Spoiler: [http://www.fanfiction.net/] replace_metadata: characters=>Sam W\.=>Sam Witwicky Spoiler: [http://www.fanfiction.net/] replace_metadata: Sam W\.=>Sam Witwicky I am not sure what I am doing wrong, but I cannot get Sam W. to change to Sam Witwicky. - Jade Aislin

10-20-2012, 05:16 PM	#950
JimmXinu Plugin Developer Posts: 6,312 Karma: 3966249 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	Each replace_metadata line is checked in order for regexp and conditional match. So the first one will be applied. However, if the changed value matches a later line, that later line will also be applied. A completely made up example: Code: replace_metadata: genre=>Sci-Fi=>SF SF=>San Fransisco Sci-Fi would be converted first to SF, then SF to San Fransisco. Reversing the order of the two lines would prevent that in this case.

10-22-2012, 07:13 PM	#954
JimmXinu Plugin Developer Posts: 6,312 Karma: 3966249 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	New test version with additional metadata for www.fimfiction.net. New entries are: likes,dislikes,views,total_views,short_description Added to defaults.ini (for reference, you don't need to add these to personal.ini unless you want to change them.: Spoiler: Code: [www.fimfiction.net] ## Site dedicated to these categories/characters/ships extracategories:My Little Pony: Friendship is Magic ## Extra metadata that this adapter knows about. See [dramione.org] ## for examples of how to use them. extra_valid_entries:likes,dislikes,views,total_views,short_description likes_label:Likes dislikes_label:Dislikes views_label:Highest Single Chapter Views total_views_label:Total Views short_description_label:Short Summary To add these to the titlepage inside the ebook, use extra_titlepage_entries or titlepage_entries. To set these to calibre custom columns, use custom_columns_settings. This version also adds a check to prevent infinite recursion if a replace_metadata conditional refers to the same entry it's trying to change. Last edited by JimmXinu; 10-23-2012 at 05:04 PM. Reason: Remove obsolete beta versions

10-22-2012, 07:51 PM	#955
crackshot91 Enthusiast Posts: 40 Karma: 98 Join Date: Jan 2011 Device: Kindle 3G	EDIT: OH GOD I'M NOT VERY SMART. I had "custom_column_settings" instead of "custom_columns_settings" in the override section of the ini. Please ignore my stupidity. ANOTHER EDIT: I was wondering if there was a way to get a custom column to report the ratio of likes:dislikes? What I'm working toward is being able to sort first by total views, and then by likes/Dislikes ratio to approximate fimfiction's rating system (And generally just browse my catalog easily by rating.) What you've done so far is wonderful, so I don't want to push it. ANOTHER EDIT X2: Gah. I feel like I'm being a nuisance. I've added "dislikes=>#dislikes,r total_views=>#views,r" to the "custom_solumns_settings" in the personal ini. Now I get an error. "too many values to unpack". The custom columns entries only need to be separated by a space, right? Is it possible to have more than one custom column entry in the personal.ini? Everything below this line is silliness: ------------------------------------------------------------------------------------- Thank you for your help, Jimm! Unfortunately, I feel kinda stupid right now. I'm looking at this page of the thread. Specifically this part: Code: ## (Plugin Only) - You can also populate calibre custom columns with ## the site specific metadata using custom_columns_settings (but only ## if 'Allow custom_columns_settings from personal.ini' is checked in ## the plugin GUI config.) There are three parts, the entry name, ## then the label of the calibre custom column, then (optionally) a ## 'mode'. 'r' to Replace any existing values, 'a' to Add to existing ## value (use with tag-like columns), and 'n' for setting on New books ## only. ## Make sure to keep at least one space at the start of each ## line. custom_columns_settings: cliches=>#acolumn,r themes=>#bcolumn,a timeline=>#ccolumn,n But I can't seem to get the custom columns to work right. If I understand correctly, if I wanna have a column for likes, I need: custom_columns_settings: likes=>#Likes,r Is that right? What is meant by the "Label of the calibre custom column"? Is that the lookup name? Or the Column Heading? Because I've tried both. The lookup name for the likes column is "likes" and the column heading is "Likes". I don't think I'm doing this right. Just to check: the likes column needs to be an integer column, right? Format: {0:} Also, I have FFDL set to allow custom_column_settings to overwrite. Last edited by crackshot91; 10-22-2012 at 08:13 PM.

10-23-2012, 05:03 PM	#958
JimmXinu Plugin Developer Posts: 6,312 Karma: 3966249 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	Version 1.6.15 - 23 Oct 2012 Fix for some stories on hpfandom.net Fix for some stories on fimfiction.net Add conditional replace_metadata feature Extra metadata for fimfiction

10-22-2012, 02:46 PM	#951
crackshot91 Enthusiast Posts: 40 Karma: 98 Join Date: Jan 2011 Device: Kindle 3G	Hey Jimm. Another question for ya. Is there a way to have a column that displays the rating of a fic from fimfiction? Not the content rating, but the thumbs up/thumbs down rating. I'm not sure how to go about creating the column or even if FFDL downloads the quality rating as metadata. Thanks!

10-22-2012, 03:06 PM	#952
JimmXinu Plugin Developer Posts: 6,312 Karma: 3966249 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	@crackshot91 - I can add a new site specific metadata for it if you can give some example URLs. I don't read that site, so I don't know what they usually look like. Ideally, I'd like to see a story with each rating and one without a rating (if that can happen).

10-22-2012, 08:53 PM	#957
crackshot91 Enthusiast Posts: 40 Karma: 98 Join Date: Jan 2011 Device: Kindle 3G	Oh god thank you. I put the custom columns settings entries on separate lines. Works great now. Also, composite columns seems like a good way to go. I'll give that a try. And thanks for the note about the lack of edit notifications. I had a feeling that was the case, but I really didn't want to spam the thread. I see now that it may have been necessary. Imagine if you saw my post before the edits! D: Thanks again for your help! You rock!

10-24-2012, 03:12 PM	#959
thecookiemommma Enthusiast Posts: 30 Karma: 7286 Join Date: Mar 2011 Device: ZTE Grand Max + / HP Slate 7 HD	Thank you, Jimi for all your hard work. I use this all the time, and it seems to be pretty darn awesome. It's made collecting fic to read nearly seamless. Thank you!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Count Pages	kiwidude	Plugins	1750	04-07-2024 01:20 AM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 08:39 AM
[GUI Plugin] Resize Cover	kiwidude	Plugins	95	03-16-2024 11:55 PM
[GUI Plugin] Find Duplicates	kiwidude	Plugins	1096	03-16-2024 11:28 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM