calibre and devices questions

[artisticforge]

hello;

having determined with griker's help why calibre did what it did i have several questions.

how does calibre 'know' which periodicals have all ready been uploaded to the devices?

assuming that calibre stores this information somewhere, where does calibre store this information?

i assume that calibre assumes that there is only one device that it is uploading to automatically. is it correct to assume that calibre does not make any differentiation between devices, there is no way to have calibre upload periodicals to the 32GB iPod but not the 64GB iPod nor the 16GB iPod.

those are my questions for now.

[kovidgoyal]

calibre simply uploads news to whichever device is first connected after the news download completes.

[artisticforge]

hello;

thank you, that is what griker and i had assumed.

how does calibre remember which periodicals have been uploaded and which have not been uploaded to a device?

where does calibre store the information concerning the periodicals and devices?

[kovidgoyal]

every time news is downloaded its id in the calibre db is stored in a config file. Then when a device is connected, the news is sent and the id is deleted from the config file.

[artisticforge]

Originally Posted by kovidgoyal

every time news is downloaded its id in the calibre db is stored in a config file. Then when a device is connected, the news is sent and the id is deleted from the config file.

would you be willing to provide the name and location of the config file?

[kovidgoyal]

IIRC the name is dynamic.pickle and the location is easily accessed from Preferences->Advanced

[artisticforge]

hello;

i have not programmed in python before so it is taking me time to read and understand the source code. slowly but surely i am beginning to understand the 'class DynamicConfig'
. concerning ~/Library/Preferences/calibre/dynamic.pickle (this is on macosx) is there any documentation concerning the structure of the file. using 'hexdump -C dynamic.pickle' i am able to determine some of the structure. to be honest it is strictly hit-n-miss.

thank you for your replies.

[kovidgoyal]

Its a python pickle file (persistent object storage). Use the pickle module in python to read its contents

[artisticforge]

hello;

i have been attempting to use the pickle module and have come to the conclusion that while i am calling it correctly the dynamic.pickle file itself is causing python to search for the module calibre.utils.config . from what i am able to figure out, in the very beginning of the dynamic.pickle file is the name of the module and defined class that was used to create the pickle file. in this case that is calibre.utils.config.DynamicConfig .
to be able to use python to unpickle the pickle file i need to create a 'package' of the necessary python code from calibre and install it in the python sys.path so that python is able to find the module 'calibre.util.config' which will be used to unpickle the dynamic.pickle file. python appears to have the idea that the module and class that wrote the pickle file should be used to unpickle the pickle file.

the alternative of using python would be to use perl or straight c programming to unpickle the dynamic.pickle file. the disadvantage of doing so is not knowing what the structure of the dynamic.pickle file is. i am then back to hit-n-miss.

the upside is that after years of avoiding it i am learning python.
i am learning why not to use 'canned' routines to save configuration information.

anyway, i will keeping plodding forward.

[kovidgoyal]

Use

calibre-debug

[itimpi]

Originally Posted by artisticforge

i am learning why not to use 'canned' routines to save configuration information.

the pickle file is just a serialization of a Python object which is why the original class file is needed to make sense of it. The advantage from a developers point of view is that no special code is needed to read/write the configuration information as one simply re-instantiates the original python object.

Exactly the same issue would apply in any language that allow objects to be serialized and then reloaded.

[chaley]

I am not sure what you want to do, but it seems like you want to start by seeing what it in dynamic.pickle. The following code will do that.

Create a file foo.py containing

import pickle

stuff = pickle.load(open('dynamic.pickle', 'rb'))
for k in sorted(stuff.keys()):
    print '%s: %s'%(k, stuff[k])

Copy the dynamic.pickle file you want to examine to the same directory as xxx.py (or change the path in the code).

From a command line, run

calibre-debug -e foo.py

The content will be printed as key: value. Some of the values are binary, so will print strangely.

When I run it, the first few lines of the output are

BooksViewcolumn width map: {u'rating': 80, u'tags': 258, u'series': 120, u'title': 192, u'authors': 184, u'ondevice': 70, u'timestamp': 69}
BooksViewcolumn widths: (232, 195, 140, 87, 79, 237)
DeviceBooksViewcolumn width map: {}
DeviceBooksViewcolumn widths: (340, 102, 88, 69, 380)