KISS for New calibre Users

[unboggling]

Originally Posted by kiwidude

...I remember seeing a post from you that you were revisiting your tag approach and I thought "cool, maybe he is going to be more conventional in his recommendation". Then I saw your post above and it seems you just traded some symbols in your tags. So I wondered if it was going to feature in your guidelines like it once had when I first saw it. Your explanation above is good on your intentions with that.

Much kudos to you for your continued efforts on the iterations.

Thanks for the kudos. I'm trying to sync actual workflow iterations with revision iterations and it's really not very easy to do that without some lag.

Re the "your recommendations" or "your guidelines" thing, it's really not supposed to be either. Not even in disguise. So maybe I should rethink this thing eventually becoming a sticky, and just keep things as they are in this thread. All I want to do is document what I'm doing so new users might find some of it useful, in learning their own ways to do things by seeing examples another users' ways. I'm not intending to recommend or guide. Except, of course, to KISS. Which it seems I'm not very good at myself.

[kiwidude]

@travger - there is an easier way without sacrificing readability. Just go to Preferences->Searching, and make sure that "comments" is not one of your searched fields in the "Limit the searched metadata". I petitioned for this feature many aeons ago for precisely this reason and chaley was kind enough to implement it.

In my case I was getting grumpy with typing an author name and having books come back in searches which were unrelated just purely because that name was mentioned in comments for some other book. Now that is no longer an issue - my searches only cover the "tittle, authors, tags, series" columns. And if I ever do need to search comments (which I never do) I still can - I just type comments: before the search term.

BTW please don't feel like you have to justify using it the funky tag approach - as I said to unboggling if it rocks your world then go for it. However I was just suggesting it is not a "recommended practise" to newbies which is where unboggling is targeting his guide. In around a year of using the Calibre forums daily this is the only thread that I have ever seen such an approach discussed (beyond some single digit tags to support earlier versions of the catalog feature). Hence I consider it a "niche" rather than a recommendation - but hey if it works for you, go for it .

[unboggling]

Originally Posted by travger

Example:

I want to find novellas.
One of the books has "author of award-winning novella" in comments (though I try to avoid this, but...).
Now this book is also included in the search result.

Having something unusual instead of 'novella' avoids situations like this. Nvella or *nlla is my own creation.
Also, I am 98% sure of never having to explain. Nobody sits behind my computer without me hanging around with watchful eyes. If I share over the net - recipient is free to delete/change all the tags.

I haven't been tracking novellas or novelettes separately, just lump them in with short stories or short articles as %shr.

I stayed away from using asterisk or plus as symbol because they're often used as a wildcards in search and replace in calibre and other applications. Minus and plus and ampersand also, used sometimes as operators in some applications. The ones I listed above seemed relatively safe and avoided needing any special fonts or extravagant keyboard commands such as preceding keystroke with function, control, option, alt, command, or also combining one of more of those with shift key.

The ones that seem to work safely in calibre at the moment are underscore, percent, dollar, semicolon, exclamation point, at (@). I imagine the pound sign would work okay too. To make something unique, it helps to give that symbol a unique letter combination. Such as $xHFP being format problem with headers/footers/pagenumbers, where the x means it's serious enough to trash if it can't be fixed or replaced. The $x combo makes it unique no matter what else is in the tag.

Those symbols also can be used as suffixes rather than prefixes, but prefixes are usually more useful for sorting purposes.

Other symbols used in calibre regex (like plus or asterisk) could maybe be used with a backslash exception, but I'm not sure yet exactly how the search routine parses those, and why risk forgetting the backslash (if it works at all) or bothering with those at all?

[kiwidude]

@unboggling - the difference is in how you use them. Your use of symbol prefixes appears to be as a "shortcut" in the search bar.

I however never expected to type just "*" or "-" in the bar, I always prefix them like tags:- in the rare circumstances I need to refer to them. In my case their purpose is purely to reduce the need for another custom column for something I know only a small % of the books I have will need it set on, so a tag (that sorts to the left by being a symbol) works well for this purpose. As I have mentioned above the precedent for this was set by some other features in Calibre like Catalogs, which used a "+" tag to indicate a book you had read along with some other "secret tags" it supported. I don't use catalogs myself but I think nowadays it is likely all customisable anyways.

[travger]

@kiwidude - I don't yet know if I want to exclude comments, though your example with authors is a good one. It means I have to start tags like 'werewolf', 'vampire', 'magic' and whatever. I don't like having lot of tags. If I want to find all alien telepath yarns, seems like I'd get better results if I keep searching the comments too.
I'll consider it for a few years.

[unboggling]

Originally Posted by kiwidude

@unboggling - the difference is in how you use them. Your use of symbol prefixes appears to be as a "shortcut" in the search bar.

I however never expected to type just "*" or "-" in the bar, I always prefix them like tags:- in the rare circumstances I need to refer to them. In my case their purpose is purely to reduce the need for another custom column for something I know only a small % of the books I have will need it set on, so a tag (that sorts to the left by being a symbol) works well for this purpose....

Shortcut, yes, actually a short-short-cut. and btw, the abbreviations aren't necessary for shortcut to avoid typing fieldnames, colon, and "s. A prefix containing a unique symbol and letter, or symbol and number, combo would work as a search shortcut for spelled out human words too. Such as, instead of !sf or !fn:

(Edit: Sorry, forgot to surround strings with parens if they had a space in them. So put parens around each separate of the following tags containing a space)

!g science fiction OR !g fantasy

if the actual tags are (without quotes): "!g science fiction" or "!g fantasy".

So tag types could each be set up as different combos of 1 symbol with 1 letter. Though I haven't seen a need to get that formal with the tag scheme yet, if necessary I could move to something like this, either without abbreviations or with abbreviations following the symbol combo in the tags, with a space between prefix and tag being optional if applied consistently:

_f(space) or $f(space) indicating Formats.
_fq(space) for Format Quality rating, sorting it to the beginning.
$fx(space) for format problems, sorting those after the quality rating.

!g(space) indicating Book Genres.
!g1space for Primary Genre
!g2space for others, sorting them nicely after the primaries(s).

!k(space) indicating Book Kinds like Short Stories or Omnibuses.

!a(space) indicating Awards. (!a instead of @ which I presently use).

!s(space) indicating SeriesStatus and/or other statuses.

((
Edit: for example, this works great to find short science fiction:
(!g sf) and (!k shr)
or without internal spaces:
!gsf and !kshr
))

The list could go on. Like I said I haven't gone that formal with it. But the merits of a convention like that are: quick and precise informal search without needing fieldnames, with only having to enter 2 extra characters (3 with space) on each tag at initial tag entry (or renaming) to enable it. The merits of using abbreviations instead of full words for the body of the tag are: fast keyboard entry of tags originally and whenever doing searches, and seeing everything at a glance in all "narrow" columns on screen at once, vs wide columns hanging off-screen to left or right or narrower columns needing to scroll horizontally within a column.

I'm not talking about a lot of tags, either. Even when I had over 10,000 books, my combined tag list in Tags column and all Custom Columns was less than 250 tags rapidly dwindling nearer 200 over time, expected to get to 100 to 150 total eventually as I pared things down - if I hadn't trashed those books and started over, allowing an opportunity to rethink everything.

[unboggling]

Originally Posted by kiwidude

@travger - there is an easier way without sacrificing readability. Just go to Preferences->Searching, and make sure that "comments" is not one of your searched fields in the "Limit the searched metadata". I petitioned for this feature many aeons ago for precisely this reason and chaley was kind enough to implement it.
.

Nice feature. Sometimes I want to search comments, sometimes not.

[unboggling]

Originally Posted by travger

@kiwidude - I don't yet know if I want to exclude comments, though your example with authors is a good one. It means I have to start tags like 'werewolf', 'vampire', 'magic' and whatever. I don't like having lot of tags. If I want to find all alien telepath yarns, seems like I'd get better results if I keep searching the comments too.
I'll consider it for a few years.

I'm planning to do away with most of my secondary genre tags such as vampire, shifter, ghost, fae, magic, etc. Like you said, searching comments saves a lot of work and if doing that instead misses something, oh well.

[unboggling]

Originally Posted by kiwidude

...as I said to unboggling if it rocks your world then go for it. However I was just suggesting it is not a "recommended practise" to newbies which is where unboggling is targeting his guide. In around a year of using the Calibre forums daily this is the only thread that I have ever seen such an approach discussed (beyond some single digit tags to support earlier versions of the catalog feature). Hence I consider it a "niche" rather than a recommendation - but hey if it works for you, go for it .

On this verbose day I'll take this opportunity to repeat: I'm not writing a guide. I'm not making recommendations. I'm just documenting what I'm doing because I thought new users might learn something from some of it, by having some concrete examples of how one person does things. That's all. Not a user guide. Not recommendations. Except, as a sort of general rule of thumb, to KISS things to make them better.

Edit: And also, the whole process of thinking about how I use calibre (or how other people do), testing different things, and writing it all out - helps me learn new things faster….

[kiwidude]

Originally Posted by travger

@kiwidude - I don't yet know if I want to exclude comments, though your example with authors is a good one. It means I have to start tags like 'werewolf', 'vampire', 'magic' and whatever. I don't like having lot of tags. If I want to find all alien telepath yarns, seems like I'd get better results if I keep searching the comments too.
I'll consider it for a few years.

I understand what you are saying, and if searching across my Calibre library in that way was important to me I would be trying to find a way to make it work too. The problem I have with trying to do this though is that the quality of the search results is only as good as the data you have in your library. There is no guarantee that a book about vampires is going to have vampire somewhere in the comments - in fact I would suggest most of the time it won't from the books I have seen. That is - unless you add the data there yourself of course. And if you are adding data yourself, then there can be advantages to putting it into the tags field (such as ease of searching to allow access via the tags browser etc).

It is a bit like a fun "debate" Starson17 and I have had a few times about the value of the "random tags" that you get if you take in tags from Google Books. His argument is that it is quite cool to be able to do a search for some arbitrary criteria like a character name or location and have matches come back. Now I agree that would be very cool - if it "worked" properly. But it is garbage in, garbage out. There is no quality control over the tagging process (just like there is no consistency or rules over what a publisher/author puts in the synopsis for a book), hence it is down to dumb luck as to what search results if any you will get back. For his purposes, some results are better than none, so it works great for him. For me who has rather too many OCD tendencies, getting only "some" at best is not useful.

All of which means that if I want to search about a *subject* (that I know I haven't specifically tagged), I am far more likely to google specialist websites for recommended books on that subject (and then cross-reference with my library) than I am to search the comments on my Calibre books.

But I am just playing devil's advocate here. If you are happy with your search results then go for it. For myself it drove me mental to quickly search for an author like Lee Child by typing "child" and having many hundreds of results being returned because they had a word like "children" in their comments, or because Lee Child had made some soundbite as promo for some other unrelated book. I rarely type the search prefixes (too slow/clumsy to type) and don't like having flow interrupted with popup dialogs as you do for advanced search. So turning off default searching on fields other than the ones I actually want to search rocked my world

[unboggling]

Originally Posted by kiwidude

I understand what you are saying, and if searching across my Calibre library in that way was important to me I would be trying to find a way to make it work too. The problem I have with trying to do this though is that the quality of the search results is only as good as the data you have in your library. There is no guarantee that a book about vampires is going to have vampire somewhere in the comments - in fact I would suggest most of the time it won't from the books I have seen. That is - unless you add the data there yourself of course. And if you are adding data yourself, then there can be advantages to putting it into the tags field (such as ease of searching to allow access via the tags browser etc).
...
But I am just playing devil's advocate here. If you are happy with your search results then go for it. For myself it drove me mental to quickly search for an author like Lee Child by typing "child" and having many hundreds of results being returned because they had a word like "children" in their comments, or because Lee Child had made some soundbite as promo for some other unrelated book. I rarely type the search prefixes (too slow/clumsy to type) and don't like having flow interrupted with popup dialogs as you do for advanced search. So turning off default searching on fields other than the ones I actually want to search rocked my world

Hmm. On second thought, I guess I'll keep my secondary genre tags for awhile. I just noticed today that recently, after creating all my new custom columns to receive tags moved out of tags column, I'd been searching the fields: authors, series, title, tags (tags now mostly empty), and specific custom columns #dlgroup (source), #fqr (format quality), #genre, #kind, #prizes, and #status. I'd forgotten to add comments to the list, and so in addition to making searches less precise, I also suspect searching comments will slow the searches down a little. Which I vaguely recall is one reason I started restricting searches originally to just certain fields indicated in Preferences/Search.

[DoctorOhh]

Originally Posted by travger

Please can you clarify? What restrictions exactly? Examples? How can I treat for example all short stories as if in another library?

The first thing you would do is create a search for the tag - Short Stories tags:"Short Stories" then you save this search by giving it a name in the saved search window (top right) I called this saved search Short Stories, then click the white cross in the green block.

You now have a saved search called Short Stories. Now you can go to Restrict to window (top left) and choose the saved search Short Stories. Your library is now restricted to Short Stories. The number of books will be highlighted in yellow indicating a restriction is in place. You can do whatever you wish within this restricted library just as you did for the full library.

To undo the restriction go back to the Restrict to drop down window and choose the blank entry at the top. The yellow highlighting goes away and it says (all books) next to the window.

I have my library start up with a restriction titled Main Library, this is all of my books except those books tagged with both Adult and Non-fiction and any book with the tag of working. In order to start you library with a restriction go to Preferences - Behavior and choose which restriction you want in place. The content server can also be set up to run a restricted library.

I also automatically add the tag working (set under Preferences - Adding Books) to every new book added to my library and have a restriction for my working library.

[unboggling]

Thanks for explaining that, dwanthny. I've never tried to use the Restrict to box before, except with the plugin Find Duplicates, to escape the dupe list back to the entire list. What you're saying makes a lot of sense, to make the books in storage status one restriction based on a save, and have another restriction for books of the status being-processed that were just Added. So to switch booklist from one restricted list to another or back to entire library, just click something on that Restrict to menu.

So, what's the difference between a restriction and a saved search? Besides the choice in Preferences to open calibre on a booklist of a specified restriction? Is there an advantage to using Restrictions over and above using Saved Searches?

Edit: Just found one answer in the manual. Can apply further searches within a restriction, just limited to the subset of books within that restriction.

[DoctorOhh]

Originally Posted by unboggling

So, what's the difference between a restriction and a saved search? Besides the choice in Preferences to open calibre on a booklist of a specified restriction? Is there an advantage to using Restrictions over and above using Saved Searches?

Edit: Just found one answer in the manual. Can apply further searches within a restriction, just limited to the subset of books within that restriction.

Also the tag browser is now limited to show only the data from the restricted library. Essentially the books are now segregated into their own little library. When I remove the working tag the book moves from my Restricted Working library to my Restricted Main Library without moving on my disk drive.

[unboggling]

Originally Posted by dwanthny

Also the tag browser is now limited to show only the data from the restricted library. Essentially the books are now segregated into their own little library. When I remove the working tag the book moves from my Restricted Working library to my Restricted Main Library without moving on my disk drive.

Thanks.

I set up restrictions based on search on tags:"=_New", meaning it needs metadata work, and a search on not tags:"=_New", meaning everything else (Main/storage/processed books).

(I already had a column named #wkg and didn't want to have any confusion between that "working" and the needs-metadata-edit tag "working".)

It's important to remember when in a restriction that searches won't find anything outside of the restriction. I tend to forget I'm in a restriction and get startled when a search doesn't find everything I expect, then remember I'm in a restriction. For example, the command Similar Books by Author (alt A), works only on books within restriction. To avoid that for the purpose of working on newly added books, I'll probably use saved search "New" and saved search "Main" rather than restrictions.