Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 08-24-2013, 03:10 PM   #1651
Snowman
Connoisseur
Snowman doesn't litterSnowman doesn't litter
 
Posts: 97
Karma: 196
Join Date: Aug 2008
Location: London UK
Device: iPhone 5, Kindle K3
That's good to hear. Bug hunting is so much easier with something tangible to work with.

And thank-you again for such a useful plugin.

Snowman
Snowman is offline   Reply With Quote
Old 08-24-2013, 03:51 PM   #1652
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,722
Karma: 510181
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by Snowman View Post
That's good to hear. Bug hunting is so much easier with something tangible to work with.
Very much so. I've figured out what's happening, and I have a suspicion as why.

You're ending up with tags Fanfiction and FanFiction (note the second f/F). calibre 1.0 doesn't like that.

I've opened a calibre bug about it. The attached screen shot shows how to cause it outside FFDL.

Your existing books have tag Fanfiction. I suspect you changed the tag FanFiction to Fanfiction at some point across your library. In past, calibre silently folded them together, but now it's not.

In the meantime, if you go into your personal.ini and add:
Code:
[defaults]
extratags: Fanfiction
...that will change it to Fanfiction for you.

Unfortunately, I can't promise that the same fix will work for everyone, because if you have other tags in your library that have the same letters but different case as the sites you're downloading from, you'll still have the same issue.

I'm looking at ways to work around the bug in FFDL.
Attached Thumbnails
Click image for larger version

Name:	Capture.PNG
Views:	45
Size:	99.3 KB
ID:	109959  
JimmXinu is offline   Reply With Quote
Old 08-24-2013, 04:15 PM   #1653
Snowman
Connoisseur
Snowman doesn't litterSnowman doesn't litter
 
Posts: 97
Karma: 196
Join Date: Aug 2008
Location: London UK
Device: iPhone 5, Kindle K3
Awesome - that did the trick. I made the amendment, then chose a random book, blanked some columns, including wordcount, then did a force update. It worked as expected.

Incidentally,I do not remember changing any tag. I rarely do; could it be that ffn itself has various versions of Fanfiction, FanFiction, etc in its own db?

What is interesting is that I then selected all book, went to bulk-edit->tag editor, and looked for Fanfiction. I expected to see both versions, but still only the one was in the total list of tags.

So, to summarise - if a tag exists somewhere, and an book update (not just ffdl) attempts to add a tag with a different case, then a partial update will (silently) occur. This is will probably be noticed during individual updates, but is likely to be overlooked during a bulk update.

I have a feeling that I saw something similar in the "backend db testers" thread...

The post in "backend db testers" is #227-230. In this case, a changed case in a tag could cause an update to unrelated books. So similar, but not the same.

best regards

Snowman

Last edited by Snowman; 08-24-2013 at 04:26 PM. Reason: Found referred post in "New database backend"
Snowman is offline   Reply With Quote
Old 08-24-2013, 04:16 PM   #1654
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,722
Karma: 510181
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Attached is a version of FFDL that works around the calibre 1.0 tag case bug.

I believe this will get around the ConstraintError issue for FFDL users until Kovid gets it fixed.

Please let me know both if it works and if it doesn't for you.

UPDATE Aug 25, 2013 - Remove obsolete beta versions

Last edited by JimmXinu; 08-25-2013 at 12:36 PM. Reason: Remove obsolete beta versions
JimmXinu is offline   Reply With Quote
Old 08-24-2013, 04:37 PM   #1655
Snowman
Connoisseur
Snowman doesn't litterSnowman doesn't litter
 
Posts: 97
Karma: 196
Join Date: Aug 2008
Location: London UK
Device: iPhone 5, Kindle K3
Quote:
Originally Posted by JimmXinu View Post
Attached is a version of FFDL that works around the calibre 1.0 tag case bug.

I believe this will get around the ConstraintError issue for FFDL users until Kovid gets it fixed.

Please let me know both if it works and if it doesn't for you.
Yes - this also worked. I removed the ini 'extratags' entry, then installed the plugin update and restarted calibre. I then tried blanking columns for several books, and both an individual book update and a multiple book update worked as expected.

regards

Snowman
Snowman is offline   Reply With Quote
Old 08-24-2013, 11:29 PM   #1656
Sefiriot
Bujavid tekikin
Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.
 
Sefiriot's Avatar
 
Posts: 137
Karma: 40366
Join Date: Aug 2013
Location: Malaysia
Device: Samsung Galaxy Note N7000
Just tested the released update to fix ConstraintError problems: seems to have worked on the anthology I forced updated-- downloaded word count is updated and custom columns are updated, no error messages shown. I'll see later if it's resumed normal service with Reading List again.
Sefiriot is offline   Reply With Quote
Old 08-24-2013, 11:35 PM   #1657
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,722
Karma: 510181
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by JimmXinu View Post
...

You're ending up with tags Fanfiction and FanFiction (note the second f/F). calibre 1.0 doesn't like that.

I've opened a calibre bug about it. The attached screen shot shows how to cause it outside FFDL.

Your existing books have tag Fanfiction. I suspect you changed the tag FanFiction to Fanfiction at some point across your library. In past, calibre silently folded them together, but now it's not.
The good news is Kovid has acknowledged and fixed the problem. You can click the bug link above to see it.

The bad news is next update of calibre isn't due out until Friday next week.

Due to the fact that FFDL *doesn't* throw and error out for the user to see (I'm not entirely sure why that is), I am going to release the version with the work around for the benefit of users who may not otherwise notice the problem.

(FYI, you don't need to have changed FanFiction to Fanfiction on every book. If you ever changed it on one book, it would have changed for all (assuming 0.9 treated that part the same), or if you already had a single book with tag Fanfiction before using FFDL.)
JimmXinu is offline   Reply With Quote
Old 08-24-2013, 11:47 PM   #1658
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,722
Karma: 510181
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Version 1.7.37 - 24 Aug 2013
Work around minor bug (1216398) in calibre 1.0.0. Tags 'A' and 'a' (for example) conflict and prevent metadata update.
JimmXinu is offline   Reply With Quote
Old 08-25-2013, 12:15 PM   #1659
DavidBrookes
Junior Member
DavidBrookes began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Apr 2013
Device: None
Quote:
Originally Posted by JimmXinu View Post
Okay, I know what the problem is. It's something we've bumped into before when sites allow HTML tags in the title.

Up until now, I've be changing it for one site as a time as it becomes a problem. This time I've gone through and fixed it for all sites. But that ends up touching the adapters for 75 different sites.

So here's a test version for people to use for a bit to make sure I didn't break anything in the process. :-)
Just to let you know that I tried again with the updated plugin and Calibre 1.0.0 - no problems now - thank you
DavidBrookes is offline   Reply With Quote
Old 08-25-2013, 02:47 PM   #1660
Jade Aislin
Groupie
Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.Jade Aislin could sell banana peel slippers to a Deveel.
 
Posts: 150
Karma: 3100
Join Date: Sep 2011
Device: Acer Iconia 500w Tablet, PRS-T1
I've been working on placing the general replace_metadata in the defaults section. However, I'm now trying to make sure the title page is cleaned up as well.

Near the beginning of the defaults section, I have this code:

Code:
add_to_extra_valid_entries:,tcategory,tcharacters,tships
include_in_tcategory:category
include_in_tcharacters:characters
include_in_tships:ships


tcategory_label:Fandom
tcharacters_label:Character(s)
tships_label:Pairing(s)
 
add_to_replace_metadata:
 #take out the big sub headings to make the title page look neat
 tcategory=>(Book/Comics\.|Crossover\.|TV\.|Movie\.|Anime-Manga\.|Cartoon\.)=>
 tcharacters=>(Book/Comics\.|Crossover\.|TV\.|Movie\.|Anime-Manga\.|Cartoon\.)=>
 tships=>(Book/Comics\.|Crossover\.|TV\.|Movie\.|Anime-Manga\.|Cartoon\.)=>

titlepage_entries: seriesHTML,tcategory,tcharacters,tships,genre,warnings,status,numChapters,numWords,rating,datePublished,dateUpdated,description,site
This was supposed to delete the hierachical grouping, making the title page cleaner.

Further down, I have my usual add_to_replace_metadata for the categories, characters, etc. Now the above code is not working as it did before.

I thought it might not be working because it was before the rest of the replace_metadata. I tried placing this replace_metadata. However when I did that, my other replace_metadata code (for the category, characters, etc) stopped working.

I then tried to change the code to tcategory_LIST, but ran into the same problem.

I'm not sure what to try next. Any help would be appreciated.
Jade Aislin is offline   Reply With Quote
Old 08-25-2013, 03:43 PM   #1661
Sefiriot
Bujavid tekikin
Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.Sefiriot writes the songs that make the whole world sing.
 
Sefiriot's Avatar
 
Posts: 137
Karma: 40366
Join Date: Aug 2013
Location: Malaysia
Device: Samsung Galaxy Note N7000
Is anyone else getting a little strange behaviour updating or adding fics already in calibre? When I update a single fic, whether anthology or not, for some reason the last_modified value on every other fic by the same author is getting changed as well, despite nothing being done to any other fic.

I updated a story and its last_modified value shows as <date>T03:01:02. The 46 other fics by the same author I have in calibre then show up with last_modified values <date>T03:01:01-- despite my not having done anything to actively change a thing about their metadata or book file! Nothing seems to be out of the ordinary in the logs. Something strange is happening here. Did I screw up updating my copy of the plugin somehow?
Sefiriot is offline   Reply With Quote
Old 08-25-2013, 04:00 PM   #1662
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,722
Karma: 510181
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by Jade Aislin View Post
I've been working on placing the general replace_metadata in the defaults section. However, I'm now trying to make sure the title page is cleaned up as well.

Near the beginning of the defaults section, I have this code:

Code:
add_to_extra_valid_entries:...
Further down, I have my usual add_to_replace_metadata for the categories, characters, etc. Now the above code is not working as it did before.
...
You have more than one add_to_replace_metadata etc in the same section(defaults)? Because while they will add *between* sections, when in the same section only one of each key word will apply.
JimmXinu is offline   Reply With Quote
Old 08-25-2013, 04:09 PM   #1663
FaceDeer
Connoisseur
FaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enough
 
Posts: 70
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
This is interesting. FFDL is hanging at "updating calibre for FanFiction stories" whenever I try updating or adding stories to libraries on my laptop (only way out is to kill the program), but it seems to be working fine on my desktop. Unfortunately I've got to head out of the house for a bit right now, but I'll play around a bit tonight with debug logging and whatnot and see if I can figure out what's causing the difference. Anyone else see something like this with the most recent updates?
FaceDeer is online now   Reply With Quote
Old 08-25-2013, 04:16 PM   #1664
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
Posts: 1,722
Karma: 510181
Join Date: Dec 2011
Location: Midwest USA
Device: Nook STR w/Glowlight, Kindle 3g, Droid
Quote:
Originally Posted by Sefiriot View Post
Is anyone else getting a little strange behaviour updating or adding fics already in calibre? When I update a single fic, whether anthology or not, for some reason the last_modified value on every other fic by the same author is getting changed as well, despite nothing being done to any other fic.

I updated a story and its last_modified value shows as <date>T03:01:02. The 46 other fics by the same author I have in calibre then show up with last_modified values <date>T03:01:01-- despite my not having done anything to actively change a thing about their metadata or book file! Nothing seems to be out of the ordinary in the logs. Something strange is happening here. Did I screw up updating my copy of the plugin somehow?
On each add/update, FFDL updates the author's 'link field' in calibre. Commenting out that code shows that that is why the last_modified date is changed for all books by that author when any of them are updated.

Considering last_modified isn't normally visible in the GUI (at least I couldn't find it except for from CLI), does it really matter? And one could argue that it is accurate.

Looking in my library at older books that haven't been updated since calibre 1.0.0, I think that's always been the behavior.

Last edited by JimmXinu; 08-25-2013 at 05:21 PM. Reason: Grammar fix only
JimmXinu is offline   Reply With Quote
Old 08-25-2013, 06:40 PM   #1665
FaceDeer
Connoisseur
FaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enoughFaceDeer will become famous soon enough
 
Posts: 70
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
Heh. I actually do make use of the last_modified date, myself. I've got a library of in-progress stories that I'm interested in reading when finished but not actively following as they're written, and since it's uncouth to use FFDL to "check for updates" on a regular basis I created a custom column that uses last_modified to flag stories that haven't been checked in certain intervals for update (with an exponentially-increasing gap between check times, so that frequently-updated fanfics get "pinged" as quickly as once a week and long-quiescent ones only get checked every six months or more). This ensures that if a story stops updating it will only be checked a few extraneous times before being automatically "given up on." It never occurred to me that updating a fanfic would result in last_modified being changed on other documents that hadn't been flagged for updating.

Since generally each author will only have one or two stories in-progress simultaneously the situation may not have occurred frequently enough to be noticeable for me, and even if it had happened it wouldn't really have made much difference (it would simply "renew hope" on a fanfic that should be heading out into once-a-year land and check it a few extra times before it goes back out there), but it's a use-case if you were curious to have one so I figured I'd mention it.

If you want to be more rigorous anyway, perhaps make it so that FFDL doesn't modify other fanfics if the update wouldn't actually change the contents of the fanfic's author link field to a new value?
FaceDeer is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Open With kiwidude Plugins 245 10-29-2014 09:06 PM
[GUI Plugin] Count Pages kiwidude Plugins 668 10-24-2014 10:13 AM
[GUI Plugin] Find Duplicates kiwidude Plugins 508 09-02-2014 07:00 PM
[GUI Plugin] Resize Cover kiwidude Plugins 34 07-13-2014 07:15 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 01:44 AM.


MobileRead.com is a privately owned, operated and funded community.