Defining a subset of library by list

[jindroush]

Hi,

I have some external list of book ids and I want to work only on this subset of books in calibre.

It can't be done by any search right now (I mean the list of ids can't be obtained by some simple search query in calibre, it's basically random)
I don't know how to convert the list to search over 'id', but I'm still afraid it won't let me enter the huge search query (we're talking hundreds+ records).

The only thing I came up with is to define some temporary boolean column, externally fill it (sloooooooow), do my work and then remove the column.

Any ideas, please?

Thanks,
J.

[kovidgoyal]

You can use calibredb for that, see https://manual.calibre-ebook.com/gen...calibredb.html

[DaltonST]

There is a tool called VR Helper Create Search Criteria for Selected Book IDs. It limits the number to about 200 to avoid a python recursion error. That tool is within the Job Spy toolbag plugin. The search syntax you should copy and paste manually is: id:123456 OR id:856942 ..... Do in batches < 200 or so.


DaltonST

[jindroush]

Originally Posted by kovidgoyal

You can use calibredb for that, see https://manual.calibre-ebook.com/gen...calibredb.html

I was probably not specific enough - I want to work with this by-id-defined subset in calibre GUI, so that this suggestion is not useful for me?

[ilovejedd]

Originally Posted by jindroush

I was probably not specific enough - I want to work with this by-id-defined subset in calibre GUI, so that this suggestion is not useful for me?

You can set the boolean via shell script using calibre commandline. It should be quick even for hundreds of books. Then open calibre gui, filter by the boolean you just set and edit as needed.

[Adoby]

It should be possible to have a script add some tag to the books in the list, using calibredb. Then you can create a virtual library for those books and have at them in the gui.

Only you can decide if it is more work creating the script or tagging the books manually. If it is a repeat job then a script might be a good idea, even if you have to pay someone to write the script. (It smells like work...)

Edit: what ilovejedd said ...

[ilovejedd]

Plenty of text editors to automate converting the list of book ids to script. I use Notepad++ on Windows. Easy to do a regex search and replace of the list and save the result as batch file.

For example (using a csv/tag-type custom column for multiple values and easy access via tag browser):

1234
5678

to

calibredb set_custom #process 1234 "edit"
calibredb set_custom #process 5678 "edit"


Notepad++
search: ^(.*)$
replace: calibredb set_custom #process \1 "edit"


Really convenient particularly when one needs to mark hundreds (or even thousands) of books.

[Adoby]

I usually use Libre Calc to create this type of batch jobs. The original list in the left column and edit using string editing functions in steps to the right. And finally export the right-most column with the commands as text to a shell script.

[jindroush]

Yep, I can do that relatively easily, when I mentioned 'slooooow' I was talking about the process or running calibredb several hundred+ times. It's about 1-2 secs per one record write, and I wanted to avoid that.
Okay, where there's no other option...

[ilovejedd]

I used to use Excel (LibreCalc is noticeably slower than Excel on Windows) but I've found NPP's regex replace to be even faster and much more versatile. Smaller footprint, too.

[jindroush]

I mean "record write" as one calibredb one-column-value-update actual runtime, not the time to create the batchfile for calling it.

[ilovejedd]

Originally Posted by jindroush

I mean "record write" as one calibredb one-column-value-update actual runtime, not the time to create the batchfile for calling it.

I was replying to Adoby re speed of Excel, LibreCalc and NPP.

As for being 'slooooow', I reckon 60 records in 1-2 minutes is still way faster than tagging manually.

Did you try the Job Spy plugin as DaltonST recommended? With the caveat that you can only do batches of 200. Personally though, I'd rather create a one-time batch file and let it do its thing on its own time than expend effort tagging in batches.

metadata.db is an SQLite database. Reckon you could always make the edits via a database editor (at your own risk, of course).

[jindroush]

I'm using sqlite cmdline tools for data extraction, but I'm kinda reluctant to write there because I'm afraid that I can somehow break it.

Regarding calibredb being way faster than manual tagging - no doubt about it and it was always out of question of doing that thing manually.

I was just unsure about overlooking something.

I think we may consider this topic closed - there are several methods mentioned now how to accomplish this task.

[BetterRed]

@jindroush - naybe you could input your list into the Import List plugin, possibly after making it into a csv file. It will do the matching on author and title, with provision for fine tuning by hand - the end result is a set of Marked books, which can be added to a Reading List.

IMO Reading Lists are a better alternative for creating arbitrary lists of books, because they don't pollute your book metadata with arbitrary classification codes (tags or similar).

If the list results from doing a content search on the library using a search tool such as File Explorer's Search, Agent Ransack, Copernic, X1 etc, you could use the Drop Search Results plugin to Mark the books.

BTW - threads are never closed at MR, unless the posting guidelines are contravened - eg advocating piracy. You never know when someone will turn up with a better solution, which may not help the OP, but could be used by folks facing the same issue at a later point in time.

BR

[kovidgoyal]

You want to speedup calibredb, either call it wth a single invocation and the full list of ids, or run the calirbe server and have calibredb connect to it. Otherwise every invocation of calibredb will require the full library be loaded into memory, which is slow.