Include location URL links in the highlights text export?

[quietbyday]

Is there any way the highlights text export function in the viewer can include URL links to each highlight location (e.g. as a Markdown link, or just the plain URL)? It would seem to make sense given the wonderful new URL location functionality for ePUB, but I can't find any setting or way to make it happen.

If not, is it realistic to self-teach myself enough Python and Calibre dev to implement this, or can I offer a bounty for someone else to implement? I gather there aren't any plugins for the Viewer, so it's somewhere in the core code (highlights.py? – I can move this to the Development forum.)

This functionality would make it possible to generate book notes and store them in most note-taking software while maintaining links back to the exact source in one's Calibre library. I know a lot of people who would love this...! And it doesn't seem to be supported by any other ePUB reader afaik.

Thanks for any insight/advice/guidance, and thank you as always for this amazing piece of software.

[kovidgoyal]

IIRC currently the export of highlights happens in either pain text or JSON. The JSON export already contains the data needed to create links, so adding it there is trivial. But I dont think you mean JSON. In plain text I am not sure, it makes sense, the link is quite ugly.

[quietbyday]

I'm thinking of a Markdown export, which would basically be the same as text, but with the URL link added in markdown format, i.e.

[link text](URL)

With a single example (from Alice in Wonderland), this is the output:

"Who are you you?" said the Caterpillar.
[2020-12-23 6:35 PM](calibre://view-book/_hex_-43616c696272655f4c6962726172795f4d696e69/1573/EPUB?open_at=epubcfi(/4/2/4/282/1:0))

In a Markdown viewer/editor (e.g. iA Writer, Bear, Typora), this looks quite tidy, with the quote followed by the timestamp, which is a link to open the book at the desired spot. I've attached a small screenshot clipping to illustrate.

I reviewed the JSON ("calibre highlights") and it does indeed seem to have all the necessary info, although I'm not sure about the hex reference for the book I see in the location URL. I would guess that's somehow equivalent to the uuid value?

I guess if all else fails, I can try to teach myself how to parse the JSON into a Markdown. But if this were a quick fix to add as a Markdown export option, I can guarantee it would be popular with the Markdown/book annotator communities. I'm already a patron, but I would gladly send an additional thank-you contribution via Paypal, as it would make my year. I've been looking for this for years and got all excited when I saw the URL functionality in this release.

Thank you for the swift reply and the patience with my ignorance. Much appreciated!

[kovidgoyal]

Well, that's easy enough: https://github.com/kovidgoyal/calibr...36a10879860e0d

[quietbyday]

This is amazing!
One possible correction: I think line 194 of annotations.py might need to read "view-book" rather than "show-book". When I make that correction to the exported text, it works fine in the markdown editors/viewers I could test, namely Bear, ia Writer, Typora and Roam Research.
This is monumental for me. I've already sent a token of my deep appreciation through Paypal, but will share the feature in note-taking forums and encourage others to pitch in as well. Again, thank you!

[kovidgoyal]

oops, yes, you are correct https://github.com/kovidgoyal/calibr...b8a2d0ff5e25df

[quietbyday]

Hello!

Now that the feature is fully available in release 5.9.0, I'm writing it up to share more broadly.

Quick question: is there a shortcut to copy a location to clipboard as an URL while in the ebook viewer?

E.g. Right now, from whatever text I want to capture the location of, I need to hit "Esc", then "Goto", then "Location", and then the "Copy" button to capture the URL.

Is there a way to bind this to a key shortcut? I looked in Preferences, but couldn't find anything obvious.

Alternatively, could I make a feature request? In the ebook viewer, add to the "selected text popup" a "cite" option that copies the selection along with a markdown link to the URL location to the clipboard, e.g. "'selected text here' –(book title)[calibre://view-book....]"

I would be happy to offer a $200 for this feature, as I use it regularly for research.

In any case, this functionality is a gamechanger for me. Thank you again!

[quietbyday]

p.s. for your reference (any feedback/correction welcome): Calibre + Roam Research = hyperlinked smart book notes

[kovidgoyal]

See https://bugs.launchpad.net/calibre/+bug/1912070

[quietbyday]

This is fantastic! I'm sending my contribution now.

When testing it out, I'm noting one quirk, which is that for some books there are one or more square-bracketed snippets of text within the URL location (specifically in the epubcfi string) that are unescaped and "break" the Markdown link, e.g. [ch1] in the following

epubcfi%28/14/2/4/2[ch1]/138/1:0%29

I'm not sure how to reproduce the bug exactly (if it is a bug). It does seem that, in all cases thus far, simply deleting the bracketed snippets solves the problem, i.e. the link is functional and still opens the ePub at the right location. I'm not sure whether that suggests a solution. In the meantime, I can simply delete by hand as a workaround in the cases where it occurs.

Thanks again. This is an amazing tool for me!

[kovidgoyal]

Why would square brackets break markdown formatting? Isnt a markdown link of the form

[text](url)

so square brackets in the url part should not break anything.

[quietbyday]

You're quite right. It does break the link in Roam Research (which I mainly use), but that appears to be an implementation issue on their end. I'll follow up there.

Those same links with square brackets within the URL worked just fine in iaWriter, Bear and Typora (the other Markdown editors I know & use).

What I found curious is that the information in brackets did not appear to be necessary for the location link to work, which suggests to my amateur self that maybe some part of the code is being too verbose. Maybe one day I'll know enough about CFI to have an idea of what's going on there.

In any case, thank you! I've sent $$ in appreciation and will share as much as I can. If I ever make it to Mumbai again, I owe you a couple of dinners at least.

[kovidgoyal]

Cool, glad to hear it. And the part in the square brackets is an optional id assertion, it makes the CFI more robust against editing.