calibredb add questions

[jindroush]

Hi,
I have two problems:

a) calibredb.exe add --library-path=F:\TXT\books\Calibre_Cze "Dean Koontz - Collection.Poetry.The.Book.Of.Counted.Sorrows - 2003.epub"

never finishes (never = few minutes) and does something to the .db. After killing the process, the ebook is in db, also copied into the tree.
I tried to find similar problems here, never found any. The ebook should not be the problem, tried the different ones. From gui it works okay.


b) Could it be possible to add something like '--fields' switch for immediate changing custom db fields on file add?

Running latest calibre 3.16, on win7 64bit eng and being happy with it ;-)

Thx,
J.

[kovidgoyal]

If you look at manual for calibredb you will see that you can specify several metadata fields using options sunch as --title, --author --identifier etc.

As for it getting stuck, I have no idea why that is, it certainly is not normal.

[jindroush]

Originally Posted by kovidgoyal

If you look at manual for calibredb you will see that you can specify several metadata fields using options sunch as --title, --author --identifier etc.

As for it getting stuck, I have no idea why that is, it certainly is not normal.

I know I can update "standard" db fields with the cmdline switches, but I need to update only one custom field.

Regarding the "unending add" operation - is there any way how to pass debug option to calibredb or having db operations logged or something like that? I can see that .db is constantly updated, also some transaction file appears there.

[kovidgoyal]

There's now ay a single add can cause the db to be continuously updated. Are you running some other calibre program accessing the db at the same time?

And no you cannot update custom field during an add, you should use calibredb set_metadata for that with the id printed by calibredb add (of course in your case since calibredb add is hanging you cannot do that).

[jindroush]

Originally Posted by kovidgoyal

There's now ay a single add can cause the db to be continuously updated. Are you running some other calibre program accessing the db at the same time?

And no you cannot update custom field during an add, you should use calibredb set_metadata for that with the id printed by calibredb add (of course in your case since calibredb add is hanging you cannot do that).

I know there are no cmdline switches for now, I'm just asking if you maybe want to add such a switch so that I don't have to do that in two steps.

Ok, so for now, I ran the command on 16:11.
I can see calibre creating:
Koontz Dean subdir on 16:11
2003 (62602) on 16:11
2003 - Koontz Dean.epub on 16:11
from that time I can hear harddrive sounds and I can see metadata.db and metadata.db-journal being constantly updated.

No other calibre process is running, just calibredb.exe. Checked that in the task manager, there are also seen tens of thousands of read/writes. Right now, 7 minutes in running, I see 85MB of read data, 42MB of written data.

The db has 24MB, ebook 100kB.

After 10 minutes of running, I killed it. Also I have to note that this is 4-core XEON machine with nothing to do and 16GB of memory, so we're not running under stressed condition here, or swapping.


HA! I ran procmon and it seems that it does something to .opf files in my library, tons of operations around them. Checked one of the changed .opfs, it seems that it somehow 'backups' the newly created custom metadata fields to them. So it means it's not unending, but extremely slow, also doing something 'automagically' without my consent. #metoo

[kovidgoyal]

Run

calibredb backup_metadata

let it finish and after that adding should be fast.

And I have no interest in adding another option to add myself, but I wont refuse a patch.

[jindroush]

Yep, the metadata backup was the culprit behind this.
Maybe there could be a message or the different behavior, because the gui does not behave in that way.

Thanks,
J.

[kovidgoyal]

The GUI does metadata backup in the background. There is no background for CLI tools. It isn't normally a problem, unless you have lots and lots of unbacked up book records.