[GUI Plugin] Kobo Utilities

[JSWolf]

Originally Posted by davidfor

What "mess"? The columns show the state of the books. If the books are no longer on the device, the status is valid. You can send the books to the device again and then send the status to restart where you were before. Or to mark the books as read. The latter I use when I get a new book in a series. If I have removed the earlier books from the device, I resend all books in the series in case I want to check something. But, I restore the device has them as read.

But, yes, I have considered adding this. I haven't bothered to because I can clear all the fields easily enough using the metadata editor. And so far the only reasons I have needed to do this has been to reset the status while testing the plugin. That makes it a very low priority for me. If I get a few people clamouring for it, maybe that will change.

I would like to be able to look at the custom columns and know that if they are blank that either no data was written to them or if there is data, that it's a book on the H2O. I don't want obsolete data still sitting in the custom columns. So if you would not mind, please add this functionality to be able to clear the custom columns for books not on the device.

[davidfor]

Originally Posted by JSWolf

Ok, I've installed the 1.8.11 beta and I've managed to get it to blow up good.

calibre, version 2.14.0
ERROR: Unhandled exception: <b>TypeError</b>:can't compare datetime.datetime to NoneType

calibre 2.14  isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.8
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && Hyphenate This! && Favourites Menu && KindleUnpack - The Plugin && Count Pages && Manage Series && Kobo Utilities && Obok DeDRM && Find Duplicates && EpubSplit && Quality Check && Modify ePub
Traceback (most recent call last):
  File "calibre_plugins.koboutilities.action", line 786, in handle_bookmarks
  File "calibre_plugins.koboutilities.action", line 1001, in store_current_bookmark
  File "calibre_plugins.koboutilities.action", line 3612, in _store_current_bookmark
TypeError: can't compare datetime.datetime to NoneType

I did a store or restore bookmarke and had store, clear if unread, Only if more recent, and Not if Finished in library all checked.

I also selected ondevice:true and then selected the entire list to update the custom fields. Only one book had any values in the custom fields.

You started this from the menu? Did you have the "run in background" option checked? Did any of the books already have the reading status stored?

[JSWolf]

Originally Posted by davidfor

You started this from the menu? Did you have the "run in background" option checked? Did any of the books already have the reading status stored?

I did not have run in background checked. Only one of the books had the reading status stored. All the others did not.

[davidfor]

Originally Posted by JSWolf

I would like to be able to look at the custom columns and know that if they are blank that either no data was written to them or if there is data, that it's a book on the H2O. I don't want obsolete data still sitting in the custom columns. So if you would not mind, please add this functionality to be able to clear the custom columns for books not on the device.

To my mind, the only column that could be considered "obsolete" is the actual bookmark column. The percent read, the last read timestamp and the rating are all things that to my mind make sense after you remove the book from the device. Getting the books marked as read in my calibre library was the primary reason I created this function. Storing and restoring the positions was almost an afterthought.

If you want to remove these details, you can search for them easily and use metadata or bulk metadata editor to reset them. I'll add it to the suggestions, but it isn't going to happen soon. But, as Kovid likes to say, patches are always welcome.

[davidfor]

Originally Posted by Terisa de morgan

I've tested "Remove Annotation Files => Empty" and I've got a fail: "Fail: Cleaning image directory" and the following log:

Spoiler:

calibre, version 2.14.0 (win32, isfrozen: True)
Se produjo un fallo al comprobar el directorio de portadas en el dispositivo: Fallo: Limpiando el directorio de imágenes

Limpiando el directorio de imágenes
DeDRM v6.1.0: In __init__
DeDRM v6.1.0: In load_resources
DeDRM v6.1.0: verdir C:\Users\Marisa\AppData\Roaming\calibre\plugins\De DRM\6.1.0
DeDRM v6.1.0: In initialize
Python function terminated unexpectedly
bad character range (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 193, in main
File "site-packages\calibre\utils\ipc\worker.py", line 133, in arbitrary_n
File "calibre_plugins.koboutilities.jobs", line 656, in do_remove_annotations
File "calibre_plugins.koboutilities.jobs", line 567, in _remove_extra_files
File "glob.py", line 27, in glob
File "glob.py", line 59, in iglob
File "glob.py", line 78, in glob1
File "fnmatch.py", line 54, in filter
File "re.py", line 190, in compile
File "re.py", line 244, in _compile
sre_constants.error: bad character range

Note: The failure text is "similar" as I'm using the Spanish version.

Could you run calibre in debug mode and post the log? From the line in error, it looks like it is a problem with the file names or path. There is probably a weird Spanish character in there The debug log should show it.

[chaley]

Originally Posted by davidfor

OK, I can see the error in the log. Is the timestamp the last read time of a book or the current time? In one file it is "2014, 12, 29, 13, 32, 26" and the other "2014, 12, 29, 14, 28, 23".

Could you try the manual, non-background store? I think it will have the same error, but it does use a different code path, so maybe it won't.

I just got an Android phone, so hopefully I can reproduce it with that. But, it might be the weekend before I can try.

You don't need to run CC to have the error.

The wireless device driver is tripping over finding a value in the field_metadata structure, something that should not be there. The problem is to find what put the value there. You can see the value that is inserted by looking at the file metadata_db_prefs_backup.json. My suspicion is that if you start calibre, do nothing, and then quit then metadata_db_prefs_backup.json will not contain any '#value#' attributes. If you start calibre, do whatever it is that PeterT is doing then quit, metadata_db_prefs_backup.json will contain something like

      "#value#": {
        "__class__": "datetime.datetime", 
        "__value__": "2014-12-22T23:47:57+00:00"
      },

Interesting: I just noticed that the metadata_db_prefs_backup.json file contains #value# attributes in three metadata entries: #kobolastread (the value above), #koboreadpct ("#value#": 100), and #koboreadloc ("#value#": "(0)OEBPS/xhtml/cover.xhtml"). Don't know if that helps.

[Terisa de morgan]

Originally Posted by chaley

You don't need to run CC to have the error.

The wireless device driver is tripping over finding a value in the field_metadata structure, something that should not be there. The problem is to find what put the value there. You can see the value that is inserted by looking at the file metadata_db_prefs_backup.json. My suspicion is that if you start calibre, do nothing, and then quit then metadata_db_prefs_backup.json will not contain any '#value#' attributes. If you start calibre, do whatever it is that PeterT is doing then quit, metadata_db_prefs_backup.json will contain something like

      "#value#": {
        "__class__": "datetime.datetime", 
        "__value__": "2014-12-22T23:47:57+00:00"
      },

Interesting: I just noticed that the metadata_db_prefs_backup.json file contains #value# attributes in three metadata entries: #kobolastread (the value above), #koboreadpct ("#value#": 100), and #koboreadloc ("#value#": "(0)OEBPS/xhtml/cover.xhtml"). Don't know if that helps.

All the columns related to storing reading position have the #value# attribute. You only have to connect the device, if you retrieve it automatically and there's any position to store. It there isn't any book to modify, the values disappear in the next file.

[Terisa de morgan]

Originally Posted by davidfor

Could you run calibre in debug mode and post the log? From the line in error, it looks like it is a problem with the file names or path. There is probably a weird Spanish character in there The debug log should show it.

Ditto, this is the log:

Spoiler:

DEBUG: 161.5 remove_annotations_files - self.device_path='M:\'
DEBUG: 161.5 remove_annotations_files - self.options= {u'device_database_path': u'M:\\.kobo\\KoboReader.sqlite', u'removeAnnotAction': 3, u'annotations_dir': u'M:\\Digital Editions\\Annotations\\', u'count_selected_books': 0, u'device_path': 'M:\\', u'annotations_ext': '.annot', u'job_function': 'remove_annotations'}
DEBUG: 161.5 QueueProgressDialog::__init__
DEBUG: 161.5 QueueProgressDialog::do_remove_annotations_queue
DEBUG: 161.5 QueueProgressDialog::do_queue
DEBUG: 161.5 KoboUtilitiesAction::_remove_annotations_job
Worker Launch took: 0.430000066757
Job: 8 Limpiando el directorio de imágenes finished
Limpiando el directorio de imágenes
DeDRM v6.1.0: In __init__
DeDRM v6.1.0: In load_resources
DeDRM v6.1.0: verdir C:\Users\Marisa\AppData\Roaming\calibre\plugins\De DRM\6.1.0
DeDRM v6.1.0: In initialize
Python function terminated unexpectedly
bad character range (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 193, in main
File "site-packages\calibre\utils\ipc\worker.py", line 133, in arbitrary_n
File "calibre_plugins.koboutilities.jobs", line 656, in do_remove_annotations
File "calibre_plugins.koboutilities.jobs", line 567, in _remove_extra_files
File "glob.py", line 27, in glob
File "glob.py", line 59, in iglob
File "glob.py", line 78, in glob1
File "fnmatch.py", line 54, in filter
File "re.py", line 190, in compile
File "re.py", line 244, in _compile
sre_constants.error: bad character range

[Terisa de morgan]

Originally Posted by davidfor

Could you run calibre in debug mode and post the log? From the line in error, it looks like it is a problem with the file names or path. There is probably a weird Spanish character in there The debug log should show it.

I'm afraid it's not a weird Spanish character

The file is:

Diaz, Lena -[The Nursery Rhyme-2]- Simon Says Die.epub.annot

If the name is:

Simon Says Die.epub.annot

it doesn't fail and it removes the file.

If the name is:

Diaz, Lena -[The Nursery Rhyme2].epub.annot

it doesn't fail but it doesn't remove the file.

[davidfor]

Originally Posted by chaley

You don't need to run CC to have the error.

The wireless device driver is tripping over finding a value in the field_metadata structure, something that should not be there. The problem is to find what put the value there. You can see the value that is inserted by looking at the file metadata_db_prefs_backup.json. My suspicion is that if you start calibre, do nothing, and then quit then metadata_db_prefs_backup.json will not contain any '#value#' attributes. If you start calibre, do whatever it is that PeterT is doing then quit, metadata_db_prefs_backup.json will contain something like

      "#value#": {
        "__class__": "datetime.datetime", 
        "__value__": "2014-12-22T23:47:57+00:00"
      },

Interesting: I just noticed that the metadata_db_prefs_backup.json file contains #value# attributes in three metadata entries: #kobolastread (the value above), #koboreadpct ("#value#": 100), and #koboreadloc ("#value#": "(0)OEBPS/xhtml/cover.xhtml"). Don't know if that helps.

Yes, that is what is happening. And the problem is my changes when the new DB API was implemented. The handling for the old API was in the wrong place and still executed the update. I suppose that I should remove that completely now.

My changes seem to be working, I'll post an update as soon as I look at the other reported problems.

[davidfor]

Originally Posted by Terisa de morgan

I'm afraid it's not a weird Spanish character

The file is:

Diaz, Lena -[The Nursery Rhyme-2]- Simon Says Die.epub.annot

If the name is:



it doesn't fail and it removes the file.

If the name is:



it doesn't fail but it doesn't remove the file.

No, not a weird Spanish character. But, if you are going to use an invalid regex for the name, you can expect problems.

I reused the code for cleaning the image directory. It uses "glob" to get the list of files. I had forgotten that it uses linux style name matching and hence "[]" had meaning.

[Terisa de morgan]

Originally Posted by davidfor

No, not a weird Spanish character. But, if you are going to use an invalid regex for the name, you can expect problems.

I reused the code for cleaning the image directory. It uses "glob" to get the list of files. I had forgotten that it uses linux style name matching and hence "[]" had meaning.

Sorry about the annoyance Brackets are easy characters for matching them later if you are going to split the name (I used them at my own version of PRS+, which showed series in my PRS-350).

[davidfor]

Attached is an update beta to fix the three reported problems:

- Error if book has "[" or "]" in the name
- Error when running store manually in foreground
- Value written to db_prefs_backup json file

They seem to be OK from my testing. Please tell me if there are still any problems.

[PeterT]

Originally Posted by davidfor

Attached is an update beta to fix the three reported problems:

- Error if book has "[" or "]" in the name
- Error when running store manually in foreground
- Value written to db_prefs_backup json file

They seem to be OK from my testing. Please tell me if there are still any problems.

I can verify that the JSON problem is solved. (And yes... I made sure NOT to be running from source with chaley's fix active).

[Terisa de morgan]

Originally Posted by davidfor

Attached is an update beta to fix the three reported problems:

- Error if book has "[" or "]" in the name
- Error when running store manually in foreground
- Value written to db_prefs_backup json file

They seem to be OK from my testing. Please tell me if there are still any problems.

It works if the book has bracket in its name (and there's a text -the report- which it isn't included in the po file). I attach the Spanish translation(slow morning )