Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 12-11-2012, 06:04 PM   #1096
Firedancer885
Occassional Beta Tester
Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.
 
Posts: 230
Karma: 3516
Join Date: Nov 2010
Location: Hungary
Device: Samsung Galaxy Tab
http://archiveofourown.org/works/237304/chapters/363763
And here is one, that's in multiple collections.
Firedancer885 is offline   Reply With Quote
Old 12-11-2012, 06:14 PM   #1097
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,824
Karma: 510215
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by Firedancer885 View Post
Jimm, I have a request. Could you make FFDL collect the "Collections" information from AO3?
Give this version a try. It adds 'collections' as a site specific metadata type.

You can add it to your title pages with extra_titlepage_entries and to a custom calibre column with custom_columns_settings in your personal.ini.

Code:
[archiveofourown.org]
extra_titlepage_entries: collections

custom_columns_settings:
 collections=>#collectionscol
UPDATE Jan 02 2013 - Remove obsolete beta versions

Last edited by JimmXinu; 01-02-2013 at 12:58 PM. Reason: Remove obsolete beta versions
JimmXinu is offline   Reply With Quote
Old 12-11-2012, 07:52 PM   #1098
Firedancer885
Occassional Beta Tester
Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.
 
Posts: 230
Karma: 3516
Join Date: Nov 2010
Location: Hungary
Device: Samsung Galaxy Tab
Quote:
Originally Posted by JimmXinu View Post
Give this version a try. It adds 'collections' as a site specific metadata type.

You can add it to your title pages with extra_titlepage_entries and to a custom calibre column with custom_columns_settings in your personal.ini.

Code:
[archiveofourown.org]
extra_titlepage_entries: collections

custom_columns_settings:
 collections=>#collectionscol
Thank you, it seems to be working fine.
Firedancer885 is offline   Reply With Quote
Old 12-13-2012, 05:16 PM   #1099
dailyalice
Geek of All Trades
dailyalice began at the beginning.
 
dailyalice's Avatar
 
Posts: 12
Karma: 12
Join Date: Aug 2012
Location: in the Midwest
Device: Kindle 2, Kindle DX, Nook SimpleTouch Glow, iPad 2
So . . . where's the best place to put cover image exclusions in one's personal.ini? In the section for specific archives? (Would make sense for dramione, as discussed before.) For all epubs? In defaults?

I think Livejournal userhead & community icons may be found in multiple archives, though I've discovered them most frequently at AO3.

Images to exclude (as cover images) pretty much anywhere they show up:
http://l-stat.livejournal.com/img/userinfo.gif
http://l-stat.livejournal.com/img/community.gif
http://www.insanejournal.com/img/userinfo.gif
http://www.insanejournal.com/img/community.gif
http://www.dreamwidth.org/img/silk/identity/user.png

(Maybe simplest to just exclude any images with "user" or "community" in the filename?)

Of course now that Livejournal offers custom userheads, it might be trickier.
Sample image URL:
http://l-files.livejournal.net/userhead/76?v=1320914333
From this story:
http://archiveofourown.org/works/467320

Though maybe just not allowing any images with "userhead" in the path would do the trick?

Here's a Dreamwidth userhead:
http://www.dreamwidth.org/img/silk/identity/user.png
Example story with it:
http://archiveofourown.org/works/534858

And once I figure out the best way to handle this in my personal.ini, what's the best way to fix the stories in my Library that have userheads as their cover art? Update the story for just metadata with metadata & cover checked? Or will I want to overwrite the whole things? Remove existing cover images first? I guess I can try it a number of ways, but figure those who dig around with this stuff on the programming side might know. (I have about 100 stories with userhead covers, I guess.)

As always, your help is much appreciated. And I figure I can't be the only fanfic reader who has run across this and might find excluding these images helpful.
dailyalice is offline   Reply With Quote
Old 12-13-2012, 07:31 PM   #1100
Niedzielan
Junior Member
Niedzielan began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Sep 2012
Device: Kindle 4
FFDL doesn't want to download stories with accented characters, (That, or the case in point is an anomaly).

The fanfic is question is http://www.fimfiction.net/story/51241/crepsculo-sombro

I get two errors, one if I try to download it by itself:
Quote:
coercing to Unicode need string or buffer, NoneType found
Or if I download it as part of a group:
Quote:
Traceback (most recent call last):
File "site-packages\calibre\gui2\__init__.py", line 376, in dispatch
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 941, in download_list_completed
TypeError: sequence item 1: expected string or Unicode, NoneType found
Niedzielan is offline   Reply With Quote
Old 12-13-2012, 07:56 PM   #1101
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,824
Karma: 510215
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by Niedzielan View Post
FFDL doesn't want to download stories with accented characters, (That, or the case in point is an anomaly).

The fanfic is question is http://www.fimfiction.net/story/51241/crepsculo-sombro
That's a problem with fimfiction.net, not FFDL.

The JSON API provided by fimfiction isn't including the title. See the API link for yourself.

Anybody have a contact for reporting problems to fimfiction?
JimmXinu is offline   Reply With Quote
Old 12-13-2012, 08:33 PM   #1102
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,824
Karma: 510215
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by dailyalice View Post
So . . . where's the best place to put cover image exclusions in one's personal.ini? In the section for specific archives? (Would make sense for dramione, as discussed before.) For all epubs? In defaults?
The precedence order for sections (using epub from dramione.org as an example) is:
  1. defaults
  2. dramione.org
  3. epub
  4. dramione.org:epub
  5. overrides

So if you put cover_exclusion_regexp in [defaults], the value from [dramione.org] (in the system ini) will override it. If you put it under [dramione.org] in personal.ini, that will override the system ini, but only for that site.

To use one cover_exclusion_regexp setting for all sites, put it under [overrides].

Quote:
Originally Posted by dailyalice View Post
I think Livejournal userhead & community icons may be found in multiple archives, though I've discovered them most frequently at AO3.
...
The problem, of course, is that the more general the exclusion, the more likely you are to block things you don't mean to. I can help you craft regexp to meet your stated desires, but I can't make promises about false positives.

Since AO3 actually has an explicit cover feature, perhaps we should put make_firstimage_cover: false under [archiveofourown.org]...

Quote:
Originally Posted by dailyalice View Post
...
(Maybe simplest to just exclude any images with "user" or "community" in the filename?)
...
Though maybe just not allowing any images with "userhead" in the path would do the trick?
...
Here's a Dreamwidth userhead:
http://www.dreamwidth.org/img/silk/identity/user.png
This regexp meets the criteria above:
Code:
cover_exclusion_regexp:(/[^/]+(user|community)[^/]+$|userhead|/img/silk/identity/user.png)
Quote:
Originally Posted by dailyalice View Post
And once I figure out the best way to handle this in my personal.ini, what's the best way to fix the stories in my Library that have userheads as their cover art? Update the story for just metadata with metadata & cover checked? Or will I want to overwrite the whole things? Remove existing cover images first? I guess I can try it a number of ways, but figure those who dig around with this stuff on the programming side might know. (I have about 100 stories with userhead covers, I guess.)
Here's the thing: On update, FFDL will never *remove* an existing cover just because it didn't find one on the site. It used to do that, and that upset the people who make their own custom covers because it would zap them.

Removing the undesired covers: use the Modify Epub PI to remove the covers from the epubs. Then use calibre's Edit Metadata ('e' key) on the whole group at once and Remove Covers (or Set from Ebooks) to get them out of calibre, too.

Then you can use Update Always (with update cover) in FFDL if you think there might be other images that could be picked up for cover. After you've added the cover_exclusion_regexp, of course. I'd tested a couple first, before doing the lot.
JimmXinu is offline   Reply With Quote
Old 12-13-2012, 10:39 PM   #1103
dailyalice
Geek of All Trades
dailyalice began at the beginning.
 
dailyalice's Avatar
 
Posts: 12
Karma: 12
Join Date: Aug 2012
Location: in the Midwest
Device: Kindle 2, Kindle DX, Nook SimpleTouch Glow, iPad 2
Quote:
Originally Posted by JimmXinu View Post

Since AO3 actually has an explicit cover feature, perhaps we should put make_firstimage_cover: false under [archiveofourown.org]...
Ah, I hadn't realized they had an explicit cover feature, but it makes sense that they would and that would simplify matters for that site. Excellent.

Quote:
Originally Posted by JimmXinu View Post

Here's the thing: On update, FFDL will never *remove* an existing cover just because it didn't find one on the site. It used to do that, and that upset the people who make their own custom covers because it would zap them.
That makes sense, I'd wondered about the behavior and was trying to suss it out for myself. Had seemed like existing covers stayed which was sometimes what I wanted and a nice feature and sometimes not what I was going for. Usually the former, though. Easy enough to remove covers by editing the metadata and/or the ePub with the Modify ePub plug-in as you mentioned (which is what I'd been doing as needed).


Quote:
Originally Posted by JimmXinu View Post
Removing the undesired covers: use the Modify Epub PI to remove the covers from the epubs. Then use calibre's Edit Metadata ('e' key) on the whole group at once and Remove Covers (or Set from Ebooks) to get them out of calibre, too.

Then you can use Update Always (with update cover) in FFDL if you think there might be other images that could be picked up for cover. After you've added the cover_exclusion_regexp, of course. I'd tested a couple first, before doing the lot.
Thanks. I'll give the cover_exclusion_regexp some tries to see how it goes. Appreciate your help! I want to learn regular expressions myself, but seem to always get sidetracked or forget what I once knew. I still know how to navigate in vi after years of not doing so very often so you'd think I could handle this.

I think I may just fix the covers by making "make first image cover" false for the ones I need to fix at AO3 for now.

As ever, I appreciate your help and love the plug-in.
dailyalice is offline   Reply With Quote
Old 12-14-2012, 01:36 AM   #1104
FaceDeer
Connoisseur
FaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enough
 
Posts: 76
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
FiMFiction just went through a major update of their underlying code, it's possible that some of the features and APIs that FFDL made use of changed or broke a bit. Here's the news post from the day of the big update. The big surge of site-breaking bugs seems to have passed now so I'm sure bug reports on more subtle problems that might be interfering with FFDL would receive attention. Knightly is probably the guy to talk to.

Possibly on a related note, I found two stories that aren't downloading correctly: Honey Pie (error message: "Error downloading Chapter: http://www.fimfiction.net/story/68507/2/honey-pie/test! Missing required element!") and Then Everyone was an Alicorn (error message: "Error downloading Chapter: http://www.fimfiction.net/story/6427.../i-was-serious! Missing required element!"). Both of these appear to be referring to chapters that aren't visible in the stories' chapter lists. One of the major bugs that appeared when the site was upgraded was "unpublished" chapters becoming visible to readers (the site lets authors hide chapters from public view) so I'd bet bits to bagels that the site's leaking references to unpublished chapters to FFDL and then refusing to let FFDL download them.

Edit: Oh, almost forgot, I wanted to thank you for pointing out the "don't process images" flag a little while back. I've got my lossy compression OCD all nice and satisifed now with fresh new cover images and illustrations. Thanks!

Last edited by FaceDeer; 12-14-2012 at 01:39 AM.
FaceDeer is offline   Reply With Quote
Old 12-14-2012, 01:45 PM   #1105
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,824
Karma: 510215
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by FaceDeer View Post
... Knightly is probably the guy to talk to.

...I'd bet bits to bagels that the site's leaking references to unpublished chapters to FFDL and then refusing to let FFDL download them.
I agree about the chapters. In both those cases, it's the last listed chapter that's inaccessible. There is a chapter_count that could be used to leave those out, but is it guaranteed that any chapters that are inaccessible are at the end?

I've posted a comment on Knightly's page about the two API problems found so far.
JimmXinu is offline   Reply With Quote
Old 12-15-2012, 12:10 PM   #1106
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,824
Karma: 510215
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Version 1.7.1 - 15 Dec 2012
  • Allow % escapes in ffnet story URLs and use larger ffnet story cover images.
  • Force bgcolor in JPG output images when source images have transparency.
  • Add 'collections' to AO3 specific metadata.
  • Clear Reject List cache when switching libraries.
JimmXinu is offline   Reply With Quote
Old 12-15-2012, 03:43 PM   #1107
ILB
Enthusiast
ILB began at the beginning.
 
Posts: 42
Karma: 10
Join Date: Jun 2012
Location: Germany
Device: Kindle 3G Keyboard
Quote:
Originally Posted by JimmXinu View Post
[archiveofourown.org]
extra_titlepage_entries: collections

custom_columns_settings:
collections=>#collectionscol
I added this to my personal.ini and created a new column "Collection" but how do I fill this column with the collection?
I tried a few things but somehow it does not work

Last edited by ILB; 12-15-2012 at 03:47 PM.
ILB is offline   Reply With Quote
Old 12-15-2012, 04:03 PM   #1108
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,824
Karma: 510215
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by ILB View Post
I added this to my personal.ini and created a new column "Collection" but how do I fill this column with the collection?
I tried a few things but somehow it does not work
You must replace '#collectionscol' with the lookup/search name you gave the column when you created it, and there must be a space in front of 'collections' on the line after custom_columns_settings:

Code:
custom_columns_settings:
 collections=>#collectionscol
JimmXinu is offline   Reply With Quote
Old 12-15-2012, 05:57 PM   #1109
ILB
Enthusiast
ILB began at the beginning.
 
Posts: 42
Karma: 10
Join Date: Jun 2012
Location: Germany
Device: Kindle 3G Keyboard
@JimmXinu
Thanks for the fast help!
ILB is offline   Reply With Quote
Old 12-26-2012, 08:38 AM   #1110
yin.
Junior Member
yin. began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Aug 2012
Device: Samsung Galaxy S
Hey everyone,
I just noticed that some stories on fanfiction.net now have something that looks like a cover - would it be possible for the downloader plugin to have an option to download and integrate that? Here's one of those stories that has it -> http://www.fanfiction.net/s/8661803/1/The-Eyes-Have-It
yin. is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Count Pages kiwidude Plugins 709 Yesterday 06:43 PM
[GUI Plugin] Find Duplicates kiwidude Plugins 514 12-25-2014 07:11 AM
[GUI Plugin] Open With kiwidude Plugins 251 12-15-2014 01:07 PM
[GUI Plugin] Resize Cover kiwidude Plugins 34 07-13-2014 08:15 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 01:27 PM


All times are GMT -4. The time now is 11:16 PM.


MobileRead.com is a privately owned, operated and funded community.