Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 02-26-2011, 03:27 PM   #1
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
User Categories remain after deleted

While testing out integrating user categories into my plugin (which works well - Chaley thanks for the tip about how to access it), I found that after I deleted a category (it disappears from the left window), I could still see information on it - even after stopping and restarting Calibre. And if I recreated the user category with the same name with no data, I could still see the info. But when I added an entry to the user category, I correctly pick up the new number of books in the category.

Is there a method I should be calling to make sure I have the current information or is this not working correctly?

I'm using something like this to read the data:
...
library_book = self.db.get_metadata(id, index_is_id=True)
if library_book.user_categories:
for c in library_book.user_categories.keys():
if library_book.user_categories[c]:
book['user_categories'].append(c)
meme is offline   Reply With Quote
Old 02-26-2011, 03:51 PM   #2
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
How are you deleting the user category? You should be calling self.gui.do_delete_user_category(category_name), an api that will check several conditions and open message boxes to ask questions if needed.

If I understand you correctly, then the category is not really being deleted. db2.get_metadata reads the stored preferences to get the current item values of the categories, then scans the book metadata to see those items are in the book. If you are seeing information after deleting the category, then the category is not really gone from the preferences.

You cannot modify a user category via the return value of get_metadata. Like all the attributes returned that way, it is read only.

There are several APIs available to manipulate user categories. Some that are usable from the GUI (might open dialog boxes) are (source for all is in tag_view.py):
self.gui.do_delete_user_category
self.gui.do_del_item_from_user_cat
self.gui.do_add_item_to_user_cat
self.gui.do_add_subcategory

Some lower-level APIs are in self.gui.tags_view.model():
rename_item_in_all_user_categories()
delete_item_from_all_user_categories
delete_item_from_user_category
chaley is offline   Reply With Quote
Old 02-26-2011, 04:47 PM   #3
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
I'm deleting the category by using the GUI - right clicking and selecting delete. I'm not using the plugin or a function call. I was testing the scenario of a user dynamically adding and removing categories to see if I picked the changes.

It doesn't look like the categories are being fully deleted from the DB - but they are definitely not shown on the GUI.
meme is offline   Reply With Quote
Old 02-26-2011, 05:09 PM   #4
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
OK, I am confused. As far as I can see, the categories are deleted.

Tested using the following program:
Code:
from calibre.library.database2 import LibraryDatabase2

db = LibraryDatabase2(r'C:\CBH_Data\calibre_development\Library.test_small')
for k in sorted(db.prefs.get('user_categories', {})):
    print k
I start the calibre gui, then run the program while the GUI is still running. The output is:
Code:
>calibre-debug -e print_user_categories.py
2
AA
AA.1
AA.1.2
bb
bb.asdasaaa
bb.b2
bb.b2.asdasaaa
bb.b2.d
bb.zzz
g
Using the still-running calibre's gui, I delete the category '2' using right-click & delete, then run the program again.
Code:
>calibre-debug -e print_user_categories.py
AA
AA.1
AA.1.2
bb
bb.asdasaaa
bb.b2
bb.b2.asdasaaa
bb.b2.d
bb.zzz
g
The category '2' is gone. Still running the same calibre GUI instance, I delete AA using right-click and delete. The result:
Code:
>calibre-debug -e print_user_categories.py
bb
bb.asdasaaa
bb.b2
bb.b2.asdasaaa
bb.b2.d
bb.zzz
g
AA and all its children are gone.

I shut down calibre and again run the program. The output is the same as the last one above. I restart calibre and again run the program. The output is again the same. I use manage categories, and the categories are not there. I re-add the category '2'. It appears, but without content.

What are you doing that is different from the above?
chaley is offline   Reply With Quote
Old 02-26-2011, 06:02 PM   #5
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
I'll run some more tests tomorrow.
meme is offline   Reply With Quote
Old 02-27-2011, 03:15 AM   #6
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
I think I might have figured out what this discussion is about.

First, let's establish some vocabulary:

- Item: some book metadata that appears in a column. A tag is an item. So is a series name, or the value of a text custom column.

- Built-in category (BC): a column (field) that appears in the tag browser. Built-in categories contain items.

- User category (UC): a pseudo-column appearing in the tag browser. User categories are not related columns in the library view.

Using these terms:

BCs contain by-value references to items, which as noted above are actual metadata values in books. BCs are rebuilt when metadata changes. There is no persistent representation of BCs other than the metadata in the books.

UCs contain by-name references to items, in the form (item value, item BC). These references are stored separately, because they cannot be re-created from information in books.

I think you have discovered that the references in the UC remain after deleting items. If you re-create the item, then it will reappear in the UC because the reference is still valid. The by-name reference in the UC to the now-gone item remains, because the reference has no connection to operations on the item. You can test this by opening manage categories after deleting an item. The UC will show the item name along with the comment (Not in any book).

References cannot be deleted when the item is not shown in a BC. Consider restrictions: it is possible for items not to be shown, but the references to them must not be deleted. It is problematic whether renaming items should modify the references. Consider swapping names for items. The operation is A-> TEMP, B->A, TEMP->B. What should happen to the references during this operation? Similar problems arise when renaming items in one book but not another and with renaming hierarchical items; it is not clear what should happen to the reference. Currently calibre does nothing, leaving it to the user to decide what to do (if anything).

References to missing items could be purged at startup or shutdown (startup is easier), although I am not sure that doing so would be the right thing to do. The problem is (again) performance. Get_categories must be called to build the necessary maps, which takes measurable time for huge libraries. It might be possible to detect the initial get_categories done during GUI startup, which would avoid the penalty.
chaley is offline   Reply With Quote
Old 02-27-2011, 04:06 AM   #7
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
I can't duplicate the exact scenario I described - but I may have found what happened. I think it might be a case issue.

In add_category gui2/dialogs/tag_categories.py you do a strcmp to see if the category already exists - but you don't ignore case like you do elsewhere (when reading in the db/appending a #, etc.) although the error message you generate says case may be the reason it exists. So the category is added to your internal list, but later on Calibre generates an unhandled exception ValueError: Attempt to add duplicate search term "@catc". So the item doesn't show in the tag browser, but is still available from the db.

I tested with a long name so perhaps I mistyped it and didn't notice the warning, etc. Right now I can add/delete and my plugin sees the changes fine except for the case issue above.
meme is offline   Reply With Quote
Old 02-27-2011, 04:19 AM   #8
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by meme View Post
In add_category gui2/dialogs/tag_categories.py you do a strcmp to see if the category already exists - but you don't ignore case like you do elsewhere (when reading in the db/appending a #, etc.) although the error message you generate says case may be the reason it exists.
strcmp ignores case.
Quote:
So the category is added to your internal list, but later on Calibre generates an unhandled exception ValueError: Attempt to add duplicate search term "@catc". So the item doesn't show in the tag browser, but is still available from the db.
I can't repeat this using manage categories, but I can repeat it using right-click rename. Is that possibly what you did?
chaley is offline   Reply With Quote
Old 02-27-2011, 04:25 AM   #9
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
Try right click -> Manage User Categories
create ACAT
create acat.stuff

If I just try to create acat I get the right warning message.
meme is offline   Reply With Quote
Old 02-27-2011, 06:39 AM   #10
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Thanks. I have fixed this both in manage categories and right-click rename.

Submitted for inclusion.
chaley is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Merge stuff in user categories with same name. silentguy Development 1 02-26-2011 07:02 AM
Early warning: addition of hierarchical user and built-in categories chaley Development 2 02-24-2011 03:15 PM
Vista user profile deleted...how to get library LightHouse Lady Calibre 4 02-18-2011 11:14 AM
0.7.3 - User Categories nynaevelan Library Management 0 02-03-2011 11:51 AM
Deleted books remain in library? jackie999 Calibre 7 10-12-2010 02:00 PM


All times are GMT -4. The time now is 11:10 AM.


MobileRead.com is a privately owned, operated and funded community.