Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 09-29-2024, 01:59 PM   #1
calibre-user
Junior Member
calibre-user began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Mar 2024
Device: Mac OS
Reflecting Calibre Tags as Mac or Windows Tags ("OS-native Tags") ?

Hello fellow Calibre users,

I've been using Calibre for a number of years now to manage all my e-books, including technical textbooks.

Lecture notes (pdf, docx) and personal technical notes, which aren't e-books, are not Calibre-ized.

As such, whenever a search for a technical concept, for example the term "homomorph*", is conducted, at least two searches have to be performed – one using Calibre's (impressive) search functionality to look through my e-textbooks, and another to search across documents outside Calibre.

For succinctness, let me call searching using methods outside of the Calibre app "non-Calibre search". This covers searching using the Window's File Explorer, Mac's Finder app, or the Foxtrot Search app, as per my habit.

(I briefly describe the Foxtrot Search app at the end of this post.)

A naïve way to search once instead of twice is to task the non-Calibre search query to also look within Calibre's e-books. This may be done by specifying the inclusion of Calibre's e-book directory in its search efforts.

When this is done, searching for the occurrence of the word "advertise" in a mix of Linear Algebra & Corporate Marketing documents works fine because "advertise" is rarely used in Linear Algebra exposition, resulting in the search only returning Corporate Marketing documents. Note that documents here refer to both Calibre-ized e-books and files outside of Calibre since the single search query was conducted over both Calibre's e-book directory and elsewhere.

However, searching for words that commonly occur in both the communities of Linear Algebra & Corporate Marketing, like "rank", "transformation", "optimization", "projection", "dimension", when one is only concerned with Linear Algebra concepts sees to search results containing irrelevant Corporate Marketing documents. This is why I described this method – the mere inclusion of of Calibre's e-book directory in a non-Calibre search effort – as naïve.

A better approach that reduces irrelevant documents turning up in the search results is to instruct the search effort to focus solely on Linear Algebra documents.

To achieve this, Windows File Explorer, Mac Finder app, and Foxtrot Search allow their users to specify filters. One such filter works with OS-native tags. To my knowledge (and I don't work in the space of Filesystems), an OS-native tag for a file is commonly reflected as an extended attribute in that file. For example, if a file were tagged as "psychology" via the Mac Finder app, running the terminal command $xattr -l prints "psychology" among other things. More precisely, for the programmers reading this, it prints a bunch of text that contains the string "psychology".

So, via a tag filter, one could instruct the search effort to only focus on documents tagged "Linear Algebra", reducing the appearance of irrelevant documents in the search results.

To my knowledge, Calibre 17.9 doesn't reflect its own tags as OS-native tags by default, motivating me to pen this thread with the hope that someone might know how to get Calibre to do so, or to hear from someone who might be able to offer other possible solutions.

To summarize, it'd be quite neat if Calibre could reflect its own tags as OS-native tags so that other search applications could filter on these OS-native tags to reduce users' search efforts. From a certain perspective, OS-native tags used in this manner represents a primitive inter-application communication.

(To clarify, by "reflect Calibre tags as OS-native tags", I mean that if the file, book.pdf, were tagged as "real analysis" in the Calibre app, this file would then have an OS-native tag added, possibly an extended attribute represented by the text "calibre: real analysis", or another text to this effect. And when the user removes this tag using the Calibre app, the corresponding OS-native tag is removed as well.)


****

Foxtrot Search (https://foxtrot-search.com/) is an application that lets users search within documents (.pdf, .doc, etc. files). It is quite sophisticated in my opinion, and has a UI that is really handy for finding textual content occurring several times in documents, across several documents, that may be scattered across many paths.

A thread was recently raised in Foxtrot's forums, seeking suggestions for filtering on Calibre's e-book path conventions: https://forum.foxtrot-search.com/ind...h=612&start=0&

Last edited by calibre-user; 09-29-2024 at 02:03 PM.
calibre-user is offline   Reply With Quote
Old 09-29-2024, 05:47 PM   #2
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,958
Karma: 27620688
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Have you tried using the Embed metadata feature, you will probably need to add it to a toolbar/menu, Q.E.D. on Windows:

Click image for larger version

Name:	Screenshot 2024-09-30 074119.jpg
Views:	8
Size:	248.9 KB
ID:	211142

BR

Last edited by BetterRed; 09-30-2024 at 09:06 PM.
BetterRed is online now   Reply With Quote
Advert
Old 09-29-2024, 07:07 PM   #3
calibre-user
Junior Member
calibre-user began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Mar 2024
Device: Mac OS
Quote:
Originally Posted by BetterRed View Post
Have you tried using the Embed metadata feature, you will probably need to add it to a toolbar/menu, Q.E.D. on Windows:

Attachment 211115

BR
First, thank you BR for the suggestion. Am pleasantly surprised at the diverse array of functionality Kovid Goyal has implemented in Calibre.

Moments ago, I ran the "embed metadata" successfully on a pdf file and was able to see the tags with the terminal command mdls (on a Mac). So, Calibre embeds the metadata into what I believe is called the XMP headers in a pdf file but not the extended attributes of a file (viewable with command xattr on a Mac).

Embedded Calibre tags aren't interpreted by Mac OS as tags ~probably~ because Calibre tags are written to a different key in the XMP header. A file's Mac OS tags show up in that file's XMP header under the key "kMDItemUserTags" while Calibre's embed metadata feature writes Calibre tags to the key "kMDItemKeywords". But the real reason might require more digging into how Mac reads tags off a file.

While my Calibre tags have not yet turned into Mac OS tags, this feels like a step in the right direction. I think I should next figure out a means of using the embedded metadata content to set the file's Mac OS tags.

Last edited by calibre-user; 09-29-2024 at 07:11 PM. Reason: added more details
calibre-user is offline   Reply With Quote
Old 09-29-2024, 08:16 PM   #4
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,958
Karma: 27620688
Join Date: Mar 2012
Location: Sydney Australia
Device: none
I'm surprised that calibre's tags don't show up in MacOS in the same way they do in Windows. Do they show up in the file, e.g.:

Word DOCX

Click image for larger version

Name:	word.jpg
Views:	8
Size:	144.7 KB
ID:	211143

xChange PDF

Click image for larger version

Name:	pdf.jpg
Views:	8
Size:	97.6 KB
ID:	211144

BR

Last edited by BetterRed; 09-30-2024 at 09:18 PM.
BetterRed is online now   Reply With Quote
Old 09-30-2024, 12:46 AM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 44,413
Karma: 23977332
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibre does not use/support extended file attributes. They are extremely brittle, platform specific, filesystem specific, dont survive roundtripping via most comman archive formats etc. You can embed metadata inside the file and presumably your search tool can be configured to filter based on that metadata.
kovidgoyal is offline   Reply With Quote
Advert
Old 09-30-2024, 03:20 AM   #6
calibre-user
Junior Member
calibre-user began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Mar 2024
Device: Mac OS
Quote:
Originally Posted by BetterRed View Post
I'm surprised that calibre's tags don't show up in MacOS in the same way they do in Windows. Do they show up in the file, e.g.:

Word DOCX

Attachment 211118

xChange PDF

Attachment 211119

BR
Hi BR, thanks once again for the helpful comment and screenshots. It's neat to see that Calibre on Windows is able to write directly to a file's Windows-native tag set.

However, a bit sadly, Calibre on the Mac doesn't seem to be able to write to a file's Mac-native tags.

Five minutes ago, I imported a newly created Microsoft Word file into Calibre 7.19 and applied two tags to it within Calibre, and ran Calibre's "embed metadata" successfully. After that, I was not able to see any tags in the Word app (File > Properties > Summary > "Keywords" field), nor with the Finder app (Finder is the Mac equivalent of Window's File Explorer).

Out of curiosity, I added a tag "test-tag" to this Keyword field in Microsoft Word to see if it would show up as a Mac-native tag – it did not, but it did show up as a keyword in the Finder app. The screenshot below shows this outcome:

Click image for larger version

Name:	Screenshot 2024-09-30 at 3.09.37 PM.png
Views:	23
Size:	142.6 KB
ID:	211127

I took a few steps back to see if Calibre's tags might also be embedded in this manner am happy to see that it does:

Click image for larger version

Name:	Screenshot 2024-09-30 at 3.16.06 PM.png
Views:	17
Size:	173.3 KB
ID:	211128

At this stage, I'm looking for some automated way of getting these keywords to be reflected as Mac-native tags.

Last edited by pdurrant; 09-30-2024 at 03:42 AM. Reason: attached images in the post.
calibre-user is offline   Reply With Quote
Old 09-30-2024, 03:44 AM   #7
calibre-user
Junior Member
calibre-user began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Mar 2024
Device: Mac OS
Quote:
Originally Posted by kovidgoyal View Post
calibre does not use/support extended file attributes. They are extremely brittle, platform specific, filesystem specific, dont survive roundtripping via most comman archive formats etc. You can embed metadata inside the file and presumably your search tool can be configured to filter based on that metadata.
Hi Kovid, glad to hear from you, thank you for taking the time to weigh in on this.

My impression of extended file attributes are similar to yours: that they're brittle, and platform-specific.

Perhaps I've made the mistake of giving an impression that Mac-native tags are implemented solely as extended file attributes. I'm actually not sure about this, especially after discovering that Mac-native tags also show up in the file's XMP headers (in addition to them showing up as extended attributes).

I am now trying to see if the search app is able to filter for tags under the "kMDItemKeywords" field, and will also see if there are means of reflecting "kMDItemKeywords" tags as Mac-native tags.
calibre-user is offline   Reply With Quote
Old 09-30-2024, 05:47 AM   #8
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 76,055
Karma: 134368292
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3
Quote:
Originally Posted by BetterRed View Post
I'm surprised that calibre's tags don't show up in MacOS in the same way they do in Windows. Do they show up in the file, e.g.:

Word DOCX

Attachment 211118

xChange PDF

Attachment 211119

BR
You attachments are invalid.
JSWolf is offline   Reply With Quote
Old 09-30-2024, 05:53 AM   #9
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,958
Karma: 27620688
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by JSWolf View Post
You attachments are invalid.
They were fine (hosted here and posted as inline thumbs) until pdurrant tampered with them - for some undisclosed reason.

BR
BetterRed is online now   Reply With Quote
Old 09-30-2024, 06:03 AM   #10
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 76,055
Karma: 134368292
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3
Quote:
Originally Posted by BetterRed View Post
They were fine (hosted here and posted as inline thumbs) until pdurrant tampered with them - for some undisclosed reason.

BR
Tell him to untamper with them so they work again.
JSWolf is offline   Reply With Quote
Old 09-30-2024, 04:00 PM   #11
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 39,879
Karma: 154464500
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Libra Colour, Lenovo M8 FHD, Paperwhite 4, Tolino epos
Quote:
Originally Posted by JSWolf View Post
Tell him to untamper with them so they work again.
Perhaps it might be better to find out why @pdurrant tampered with them. Copyrighted content comes to mind as one of several possible reasons.
DNSB is offline   Reply With Quote
Old 09-30-2024, 06:07 PM   #12
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,958
Karma: 27620688
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Why have my embedded images been removed?

BR->pdurrant
Quote:
Originally Posted by BetterRed
pdurrant->BR
Quote:
Originally Posted by pdurrant
Accidental. Somehow I managed to delete the thread instead of a post. Unfortunately, restoring the thread didn't restore the images.
I would have ctrl/del'd the screen shots so I'll have to do them again Ψ³

Done

BR

Last edited by BetterRed; 09-30-2024 at 09:49 PM.
BetterRed is online now   Reply With Quote
Reply

Tags
filter, find, path, search, tag

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Are too many single-use tags (in tags-like field(s)) bad for Calibre's performance? 01100001 Library Management 10 09-21-2022 09:39 AM
Calibre 4.21 changes <b> tags in comments to <p style="font-weight: 600">! VirgoGirl Calibre 17 08-01-2020 01:39 PM
Calibre "Tags" copy to "shelves" troll05 Calibre 5 06-21-2015 05:01 AM
An reader able to read several "genre" tags on Calibre. Zabriskie Which one should I buy? 2 10-26-2012 12:24 PM
Amazon Tags - Popular tags vs Unique tags. chrisanthropic Writers' Corner 6 09-19-2011 11:18 PM


All times are GMT -4. The time now is 11:23 PM.


MobileRead.com is a privately owned, operated and funded community.