Quote:
Originally Posted by kovidgoyal
Adding books actually happens multiple stages. First ina separate thread, the filesystem is scanned for files. Then in a separate worker process for reading metadata then once that is done the books are added to the database in the GUI thread. @Terisa: what exactly is the problem? Sorry, I'm a little too busy at the moment to read through this thread.
|
Sorry Kovid, I hadn't really explained it.
The problem here is passing info form the run method to the postadd method (the original path folder). For doing that, the run method includes a dictionary entry with name/original folder and the postadd method reads the dictionary, search the entry related to the file and get the directory. If it doesn't find the entry it assumes that it has processed it (in previous calibre versions, the postadd method was called more than once for a file). The dictionary is included in the preferences method.
In this log, I see a bunch of run call and, in the middle, some postadd calls. It looks like at some point, the postadd method gets a dictionary without some of the last entries and, when it saves it without the processed file, all the intermediate entries are lost.
The solution I think would be initialize the dictionary when calibre starts and not removing anything in the postadd method. As I don't see now more than one call to postadd method, that wouldn't be a problem. The problem would be to clean the dictionary after finishing the adding (or before the beginning). I don't know where to include the cleaning because my initial idea, init, is not valid as it looks like it is called once for file.
This only happens with big batches, there is no problem with small batches as this race situation does not happen.
If somebody has a better idea for passing information between methods, I would be glad to try it.