Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 04-27-2013, 11:41 PM   #1351
Saioko
Member
Saioko began at the beginning.
 
Posts: 23
Karma: 10
Join Date: Apr 2013
Device: Digma s602
Could someone help me to set up FFDL to get covers from Fanfiction.net?

I have tried adding those lines to personal.ini but I still can't make it to get covers. I am testing on this story, which as I checked has a cover: http://www.fanfiction.net/s/6173022/...-Sand-and-Time

[www.fanfiction.net]
## fanfiction.net's 'cover' images are really just tiny thumbnails.
## Change this to false to use them anyway.
never_make_cover: false
Saioko is offline  
Old 04-28-2013, 11:54 AM   #1352
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Saioko View Post
I have tried adding those lines to personal.ini but I still can't make it to get covers. I am testing on this story, which as I checked has a cover: http://www.fanfiction.net/s/6173022/...-Sand-and-Time
You've set it up correctly. (I assume--you also need include_images:true)

If you go to the story and click the image, the shadowbox comes up, but the image doesn't display because the image file is zero length. The larger image for that particular story is broken.

FFDL doesn't try to use the ffnet thumbnail--it's just too small to be a useful cover.
JimmXinu is offline  
Advert
Old 04-28-2013, 12:45 PM   #1353
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Version 1.7.24 - 28 Apr 2013
  • Catch errors on db.set_cover().
  • Don't strip lead/trail whitespace from replace_metadata, add feature \s->' ' in replace_metadata replacements.
  • After adding to reject list, delete only rejects, not those removed from list.
  • Offer user chance to skip update when the story URL would be changed.
JimmXinu is offline  
Old 04-28-2013, 07:59 PM   #1354
Saioko
Member
Saioko began at the beginning.
 
Posts: 23
Karma: 10
Join Date: Apr 2013
Device: Digma s602
JimmXinu, thank you, at least I know I am doing it right.
I finally got that story with cover, but I guess it just doesn't always receive image right from fanfiction.net.
Saioko is offline  
Old 05-02-2013, 05:24 PM   #1355
seabream
Enthusiast
seabream can extract oil from cheeseseabream can extract oil from cheeseseabream can extract oil from cheeseseabream can extract oil from cheeseseabream can extract oil from cheeseseabream can extract oil from cheeseseabream can extract oil from cheeseseabream can extract oil from cheese
 
Posts: 48
Karma: 1056
Join Date: Jul 2012
Location: Central Canada
Device: Sony PRS-T1, RIM Playbook & Z10, Samsung Note, MacOSX
Note to Jade Aislin and anyone else who might find it useful: On AO3, "character A/character B" is by convention not the same as "character A & character B". If you look at the FAQ on tags, "/" denotes a romantic or sexual relationship, and "&" is for friendship. Tag wranglers won't override an author's chosen tag if they don't follow, so there is mixed usage.

Also, in case you haven't run into it yet, the pipe " | " used to divide character names for characters who have multiple identities e.g.: "Snow White | Mary Margaret Blanchard" from "Once Upon a Time (TV)" doesn't actually have leading and trailing spaces, though it displays with them on AO3, so if you want to replace them with something else (I use a lower case L with leading and trailing spaces because it looks almost right) because Calibre divides tags with pipes in them into separate tags, the replace metadata will fail if you include the spaces. So use e.g.:
Code:
 ships=>(\|)=> l &&category=>Once Upon a Time \(TV\)
To JimmXinu: Again, thank you for all the work you've done JimmXinu, the customization is wonderful.

Questions and comments: I found why my parentheses weren't working. I had the line
Code:
include_in_category:ao3categories
which apparently doesn't work (should have been under custom columns as
Code:
 ao3categories=>#category,a
), pre-empting most of my AO3 stuff. i.e.: everything after that.

To both re: duplicate stories on multiple sites:

I usually treat a story by the same author on a different site as a different story because different sites tag and format differently, so I can't treat the downloads the same way to get the same result anyway. If it's a new-to-me story, I pick one site's version and only update that version. I download new-to-me (or ones I think are new), with "If Story Already Exists?" set to "Add New Book" so that if I happen to end up with a unnoticed duplicate in a long list I can compare them. The times when I'll switch, or add, to my library with another site's version of an existing story are if an author has stopped updating on one site and moved to another, or I know that there have been edits, in which case again, I have two versions, one of which I'll tag "deprecated" in one fashion or another. Offhand, I can't think of a time when I'll want to overwrite one site's version with another, so I do appreciate having a notification when it might happen. I've tested 1.7.24 and the notification works on the example Jade Aislin provided.

Re: when an author or title changes
I thought that the URL was already the identifier. I remember that title used to work as the identifier, but after an update, having FFDL match two stories with the same url, one of which I'd changed the title on. I'm reasonably sure that I've updated stories that have had one or both change, and had them update successfully, showing the new title and/or author. Potentially this is an issue if you want to retain the old title and author, but they still appear in the logpage_entries if you have it set to include them.
seabream is offline  
Advert
Old 05-02-2013, 05:58 PM   #1356
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by seabream View Post
Note to Jade Aislin and anyone else who might find it useful: On AO3, "character A/character B" is by convention not the same as "character A & character B". If you look at the FAQ on tags, "/" denotes a romantic or sexual relationship, and "&" is for friendship. Tag wranglers won't override an author's chosen tag if they don't follow, so there is mixed usage. ...
Thanks for posting your findings. I don't track a lot of this metadata for my own use, so I don't see a lot of these details.

Quote:
Originally Posted by seabream View Post
Re: when an author or title changes
I thought that the URL was already the identifier. I remember that title used to work as the identifier, but after an update, having FFDL match two stories with the same url, one of which I'd changed the title on. I'm reasonably sure that I've updated stories that have had one or both change, and had them update successfully, showing the new title and/or author. Potentially this is an issue if you want to retain the old title and author, but they still appear in the logpage_entries if you have it set to include them.
FFDL matches on URL first. It only looks at title/author if no URL match was found. So that's the way I'd expect it to work--if you're updating by URL. If you select an existing book and hit 'update', it will update that book regardless of what other copies may exist.

If you don't want title/author changing in calibre, you can use the FFDL 'New Only' settings on the 'Standard Columns' tab. Inside the file (ie, titlepage/logpage), the title/author from the site will be used regardless.

(I didn't think anyone but me used logpage. )
JimmXinu is offline  
Old 05-06-2013, 05:04 PM   #1357
Jade Aislin
Groupie
Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.
 
Posts: 164
Karma: 3100
Join Date: Sep 2011
Device: Kobo Auro H2O, PRS-T1
Quote:
Originally Posted by JimmXinu View Post

FFDL matches on URL first. It only looks at title/author if no URL match was found. So that's the way I'd expect it to work--if you're updating by URL. If you select an existing book and hit 'update', it will update that book regardless of what other copies may exist.

(I didn't think anyone but me used logpage. )
I'm not sure why I thought I would get a story added twice if the author changed his or her name. For some reason, I have a dim memory of getting a story of FFN to find that I already had it under a different author name.

Re: & refering to friendship

Thanks for that info. I was not aware of the distinction. I have tried to change my code so that the relationship column will be cleaner, but I don't think I succeeded. I wanted to use the slash mark between names and then add on Friendship, Family, or One-sided, when needed.

Code:
Code:
 ships=>( )?(&|and) =>\1 (Friendship)
 ships=>(( )?(&|and|x)( )?)=>/
 
 ships=>(- brothers|-brother|Brotherly |- family| - Familial Relationship)=>\1 (Family)
 ships=>(- friendship|- frienship| friendship|-Friendship|Platonic )=>\1 (Friendship)
 ships=>(( )?[Oo]ne-sided( )?| one sided|One sided |^onesided |^Onesides |unrequited )=>\1 (One-Sided)
I had hoped that the first line would add (Friendship) to the end of the tag, instead it replaced the &. I also found at least one name in the pairing being deleted, although I haven't checked the rest of my code to find out why.


On another note, I ran into a couple errors when trying to download stories, unmatched group and invalid group reference. I'm not sure why I received these error. I didn't see any obvious similarity in the stories when I checked. Although one of the stories had the '|' in one of the fandoms that seabream mentioned. Is it a problem in my code somewhere? If so what should I be looking for?
Jade Aislin is offline  
Old 05-06-2013, 05:30 PM   #1358
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Jade Aislin View Post
I had hoped that the first line would add (Friendship) to the end of the tag, instead it replaced the &. I also found at least one name in the pairing being deleted, although I haven't checked the rest of my code to find out why.
Only the part that matches the regex is replaced. If you want to add something to the end, you'll need to match the rest of the string. Something like this should work:

ships=>( )?(&|and)(.*?)$=>\1 \3 (Friendship)

Quote:
Originally Posted by Jade Aislin View Post
On another note, I ran into a couple errors when trying to download stories, unmatched group and invalid group reference. I'm not sure why I received these error. I didn't see any obvious similarity in the stories when I checked. Although one of the stories had the '|' in one of the fandoms that seabream mentioned. Is it a problem in my code somewhere? If so what should I be looking for?
There is a known issue(look for "Brief explanation") with python's regex library not handling group matching the same as everybody else.
JimmXinu is offline  
Old 05-06-2013, 07:51 PM   #1359
KayLee
Zealot
KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.
 
KayLee's Avatar
 
Posts: 108
Karma: 518106
Join Date: May 2013
Device: Nook ST & iPad Air2
Does anybody know of a way to get stories off of WattPad with this plugin?
http://www.wattpad.com/home
KayLee is offline  
Old 05-06-2013, 08:49 PM   #1360
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
wattpad.com was previously considered for FFDL and rejected for "Entirely too much copyright." Which I confess is a bit vague--I think there was a big scary click-through copyright before.

I've looked at it again now, and the copyright doesn't necessarily look any scarier than any other now, but the viewing system is. FFDL is designed to have one chapter per page. WattPad further breaks chapters into "pages" that are loaded by javascript.

I won't rule it out quite as firmly as before, but that only means that it's moved from the "Never Likely to be Supported" list to the "Possible, but more difficult than usual" list.

This is where the list lives, btw.
JimmXinu is offline  
Old 05-06-2013, 09:50 PM   #1361
Jade Aislin
Groupie
Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.
 
Posts: 164
Karma: 3100
Join Date: Sep 2011
Device: Kobo Auro H2O, PRS-T1
Quote:
Originally Posted by JimmXinu View Post
Only the part that matches the regex is replaced. If you want to add something to the end, you'll need to match the rest of the string. Something like this should work:

ships=>( )?(&|and)(.*?)$=>\1 \3 (Friendship)
That seemed to work. Although, I changed the code to get the '/' between the names. As well as making the first space neccessary. It was accepting 'and' when it was part of a person's name, like 'Gandalf'.

New code:
ships=> (&|and)(.*?)$=>/\3 (Friendship)

However, I was only able to test it once before I started getting an 'invalid group reference' for all the stories I had already.

Quote:
Originally Posted by JimmXinu View Post
There is a known issue(look for "Brief explanation") with python's regex library not handling group matching the same as everybody else.
While I was looking up the reference you gave me, the 'unmatched group' seemed to resolve itself. However, I am still getting the 'invalid group reference'. I tried looking it up and it looked like it might be a problem with '$' when not used as an end of string reference. As far as I know my code only uses the $ to indicate the end of the string I want. Is that the only reason I would get that error?
Jade Aislin is offline  
Old 05-06-2013, 10:06 PM   #1362
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Jade Aislin View Post
While I was looking up the reference you gave me, the 'unmatched group' seemed to resolve itself. However, I am still getting the 'invalid group reference'. I tried looking it up and it looked like it might be a problem with '$' when not used as an end of string reference. As far as I know my code only uses the $ to indicate the end of the string I want. Is that the only reason I would get that error?
The "unmatched group" issue is caused when a paren grouping is skipped, but used in the replacement. (Perl, etc automatically provide an empty string in that case.) "(a)?a" will match "a", but a replacement to "\1b" will fail. So we use "(|a)a" instead, which makes the paren group empty, but not skipped.

The "invalid group reference" is more basic--you're using \3 meaning the third paren group, but you only have 2 paren groups. Change it to \2.
JimmXinu is offline  
Old 05-06-2013, 10:19 PM   #1363
KayLee
Zealot
KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.KayLee ought to be getting tired of karma fortunes by now.
 
KayLee's Avatar
 
Posts: 108
Karma: 518106
Join Date: May 2013
Device: Nook ST & iPad Air2
Quote:
Originally Posted by JimmXinu View Post
wattpad.com was previously considered for FFDL and rejected for "Entirely too much copyright." Which I confess is a bit vague--I think there was a big scary click-through copyright before.

I've looked at it again now, and the copyright doesn't necessarily look any scarier than any other now, but the viewing system is. FFDL is designed to have one chapter per page. WattPad further breaks chapters into "pages" that are loaded by javascript.

I won't rule it out quite as firmly as before, but that only means that it's moved from the "Never Likely to be Supported" list to the "Possible, but more difficult than usual" list.

This is where the list lives, btw.
Thanks for that info! also can you link me to a tutorial of how to use this plugin?
KayLee is offline  
Old 05-06-2013, 10:26 PM   #1364
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,284
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by KayLee View Post
Thanks for that info! also can you link me to a tutorial of how to use this plugin?
We don't really have one.

It's pretty simple to get started:
  1. Install the plugin(instructions on installing plugins).
  2. Choose to add it to the toolbar or where ever you like.
  3. Invoke FanFictionDL -> Add New from URL.
  4. Give a URL to a story on a supported site.
  5. Hit OK.

The stuff Jade Aislin is asking about is black magic level customization. Ignore it.
JimmXinu is offline  
Old 05-06-2013, 10:54 PM   #1365
Jade Aislin
Groupie
Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.
 
Posts: 164
Karma: 3100
Join Date: Sep 2011
Device: Kobo Auro H2O, PRS-T1
Quote:
Originally Posted by JimmXinu View Post
The "invalid group reference" is more basic--you're using \3 meaning the third paren group, but you only have 2 paren groups. Change it to \2.
That worked. Thanks.

I think I'm beginning to understand the '\'. When you use the \# in your replacement, it will place whatever is in that number parenthesis. So, \2, will place what can be found in the second parenthesis.
Jade Aislin is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Resize Cover kiwidude Plugins 95 03-16-2024 11:55 PM
[GUI Plugin] Open With kiwidude Plugins 402 03-16-2024 11:44 PM
[GUI Plugin] Find Duplicates kiwidude Plugins 1096 03-16-2024 11:28 PM
[GUI Plugin] Count Pages kiwidude Plugins 1744 03-02-2024 05:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 08:40 AM.


MobileRead.com is a privately owned, operated and funded community.