Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > ePub

Notices

Reply
 
Thread Tools Search this Thread
Old 08-06-2022, 01:25 PM   #1
DarkTrick
Enthusiast
DarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exercise
 
Posts: 25
Karma: 38926
Join Date: Jun 2015
Device: Kindle Touch
Standardizing ePub annotations

Hello,

(1) I was wondering: why is there no standard to store annotations in an ePub book? Perhaps some insights?

(2) Couldn't you write highlights and annotations directly into the document? After all it's basically XML. So you would simply need to add a tag around a passage marking the area and adding text information for the notes.
How would that work in conjunction with DRMs?


Regards
DarkTrick
DarkTrick is offline   Reply With Quote
Old 08-06-2022, 01:49 PM   #2
Turtle91
A Hairy Wizard
Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.
 
Turtle91's Avatar
 
Posts: 3,068
Karma: 18727053
Join Date: Dec 2012
Location: Charleston, SC today
Device: iPhone 11/X/6/iPad 1,2,Air & Air Pro/Surface Pro/Kindle PW & Fire
Mostly "adding to" or "changing" a copywritten publication (ePub) is a big no-no.

Besides the legal aspects, most reading apps don't want to spend the extra time, and/or overhead, to develop the functionality to store annotations in a linked document.

Having said that, there ARE some apps that allow annotations...you just need to track down which ones do it the way you want. (For example: Marvin on iOS, spent a lot of time making annotations possible)
Turtle91 is online now   Reply With Quote
Old 08-06-2022, 02:29 PM   #3
Quoth
the rook, bossing Never.
Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.
 
Quoth's Avatar
 
Posts: 10,781
Karma: 83992591
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
Quote:
Originally Posted by DarkTrick View Post
Hello,

(1) I was wondering: why is there no standard to store annotations in an ePub book?

(2) Couldn't you write highlights and annotations directly into the document?

3) How would that work in conjunction with DRMs?
1. Because it's pointless for almost anyone interested. A separate database or text file is better.

2. Madness. You could create an extra file in the epub (it's just a zip folder) for each existing file to append highlight, position, date, time and comment. But a separate file outside the epub is better. Or that the reader/app database has it and can export it as text (Kobo). Really crazy to edit the existing files inside the epub.

3. Depends on the DRM. But no copyright holder is going to agree to annotations being INSIDE the ebook. There is also no need.

4. Then there is Amazon, who have over 90% of the English Language ebook market partly due to people being conned into giving Amazon exclusivity or allowing titles in Amazon's subscription systems (usually also exclusive unless you are big publisher). Amazon won't care for your scheme and even they who control their ecosystem don't put users annotations INSIDE the ebook.
Quoth is offline   Reply With Quote
Old 08-07-2022, 09:47 AM   #4
DarkTrick
Enthusiast
DarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exercise
 
Posts: 25
Karma: 38926
Join Date: Jun 2015
Device: Kindle Touch
Disclaimer:
As letters don't convey enough information I'm not trying to discuss against you, I'm trying to figure out the rationale behind the problem I appreciate the participation!

>> (1) I was wondering: why is there no standard to store annotations in an ePub book?

> 1. Because it's pointless for almost anyone interested.

Why is that?

>> (2) Couldn't you write highlights and annotations directly into the document?

> "adding to" or "changing" a copywritten publication (ePub) is a big no-no.

> Madness.

I don't see the "madness" or problem here. If you'd had a book, you would also write directly into it - even though the content is copyright protected.


> Really crazy to edit the existing files inside the epub

Why is that?
DarkTrick is offline   Reply With Quote
Old 08-07-2022, 10:12 AM   #5
Quoth
the rook, bossing Never.
Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.
 
Quoth's Avatar
 
Posts: 10,781
Karma: 83992591
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
It messes up any indexing to write into the files. Insert also is more risky (might damage file) and slower than append. Thus create a separate file. The eBook is actually a Zip archive, so adding to it all is a more major and dangerous operation than creating separate files, which is safer, faster and no less functionality.

While it's theoretically possible to edit an ebook on an ereader, copyright and DRMed ones shouldn't be edited at all, and it's a much slower riskier thing than editing on desktop, where backups are easy. The Flash on an ereader isn't at all like NVMe SSDs.
Quoth is offline   Reply With Quote
Old 08-07-2022, 10:22 AM   #6
Quoth
the rook, bossing Never.
Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.Quoth ought to be getting tired of karma fortunes by now.
 
Quoth's Avatar
 
Posts: 10,781
Karma: 83992591
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
Quote:
Originally Posted by DarkTrick View Post
> Madness.

I don't see the "madness" or problem here. If you'd had a book, you would also write directly into it - even though the content is copyright protected.
The madness doesn't refer to annotating. Personally I'd not annotate a physical book, but you don't give or lend copies – only the physical orginal, you do own the physical medium and can mutilate it anyway you fancy, or highlight passages or add ballpoint pen annotations. But I'd hate to borrow a physical book that someone has highlighted and annotated. If I accidentally buy such a book in a charity shop I bin it. What is wonderful about ebook annotations is that they are NOT in the ebook, yet you can use them as if they are.
PDFs can have annotations (graphic image or text) added if they don't have DRM because they can be added at the end of the file and defined as an extra layer. Each annotation then has a page and position on the page.
An ebook doesn't work like a PDF. Technically it's possible to put annotations inside an ebook, but that is madness compared to a database or a per ebook separate file.
Quoth is offline   Reply With Quote
Old 08-07-2022, 10:40 AM   #7
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 12,117
Karma: 73448614
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
If you're interested, the Epub spec is maintained / managed by the Epub 3 working group https://www.w3.org/publishing/groups/epub-wg/

Remember that even if a standard were to be established there would be no guarantee that it would be implemented in devices / software; look at the number of solutions out there that don't fully implement the current standards

Sent from my Pixel 4a using Tapatalk
PeterT is offline   Reply With Quote
Old 08-07-2022, 11:57 AM   #8
Tex2002ans
Wizard
Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.
 
Posts: 2,297
Karma: 12126329
Join Date: Jul 2012
Device: Kobo Forma, Nook
Quote:
Originally Posted by DarkTrick View Post
(1) I was wondering: why is there no standard to store annotations in an ePub book? Perhaps some insights?
A lot of this was discussed back in 2017: "Merge notes into epub file?".

To my knowledge, not too much has changed on that front.

In 2021, Calibre added support for a way to export notes/highlights to markdown from Calibre's reader...

But there's definitely no cross-format, cross-compatible way of doing this. Just individuals coming up with all their non-standard, (proprietary) ways for every individual format.

Side Note: If I remember correctly, the current "best"/most compatible way, might be baking all the notes/highlights into PDFs + only read using specific apps on your computer/tablet. (That was probably discussed in the 2017 thread above.)

Quote:
Originally Posted by DarkTrick View Post
How would that work in conjunction with DRMs?
That's a huge part of it.

A lot of it also gets very complicated when the underlying file itself may change slightly. (Fixing typos, updating code, etc.)

How will your highlights and everything stay synced + transfer over?

Amazon probably has the current, best ecosystem for this... but the second something gets updated in the ebook, poof, all your notes+highlights are gone.

Quote:
Originally Posted by PeterT View Post
Remember that even if a standard were to be established there would be no guarantee that it would be implemented in devices / software; look at the number of solutions out there that don't fully implement the current standards

Last edited by Tex2002ans; 08-07-2022 at 12:05 PM.
Tex2002ans is offline   Reply With Quote
Old 08-08-2022, 07:29 AM   #9
DarkTrick
Enthusiast
DarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exerciseDarkTrick juggles running chainsaws for a bit of light exercise
 
Posts: 25
Karma: 38926
Join Date: Jun 2015
Device: Kindle Touch
Thanks again very much for the participation!

The link to the other thread and the open annotation standard were nice resources.

@Changing Epubs direclty
The problem of "how to handle annotations in a changing document" would be solved with this approach. But I understand that it's possibly dangerous. And on the technical side difficult because of the /inserting/ nature (and noted above already).

> A lot of it also gets very complicated when the underlying file itself may change slightly. (Fixing typos, updating code, etc.)
I've never thought of that (probably as I don't let my ebooks being updated. But this might indeed be a hurdle. I could imaging though, that a git-diff approach might solve this rather easily.

Anyway, I guess the conclusion for now is mostly (1) use books or (2) use a reader with a very easy notes-export functionality.
DarkTrick is offline   Reply With Quote
Old 08-08-2022, 10:16 AM   #10
rantanplan
Fanatic
rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.rantanplan ought to be getting tired of karma fortunes by now.
 
Posts: 595
Karma: 8589150
Join Date: Nov 2019
Location: Wuppertal, Germany
Device: Paperwhite (2021), Boox Note Air 2+, Kobo Clara 2E, Tolino Vision 6
Quote:
Originally Posted by DarkTrick View Post
(2) Couldn't you write highlights and annotations directly into the document? After all it's basically XML. So you would simply need to add a tag around a passage marking the area and adding text information for the notes.
How would that work in conjunction with DRMs?
Not every html file is actaully encrypted, at least with CARE-Drm. In the end you could very well define a subset of content that isn't protected. But I also think that annotations should not be part of the epub itself but maybe a standardized json file that is in the same folder or similar.
rantanplan is offline   Reply With Quote
Old 08-08-2022, 04:37 PM   #11
Tex2002ans
Wizard
Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.
 
Posts: 2,297
Karma: 12126329
Join Date: Jul 2012
Device: Kobo Forma, Nook
Quote:
Originally Posted by DarkTrick View Post
Thanks again very much for the participation!
I haven't followed it too much since that 2017 thread, but it seems like "Open Annotation" Standard has been accepted by the W3C.

You can see all the latest specs (February 2017) here:

https://w3c.github.io/web-annotation/

Also, I assume Hypothes.is has still been chipping away at this problem as well. They may be a decent resource to read up more on the latest annotation news.

Quote:
Originally Posted by DarkTrick View Post
@Changing Epubs direclty

The problem of "how to handle annotations in a changing document" would be solved with this approach. But I understand that it's possibly dangerous. And on the technical side difficult because of the /inserting/ nature (and noted above already).
The specs go into detail on:
  • Embedding into HTML or external file?
    • JSON, XML, etc.
  • How to handle morphing documents.
    • New paragraph/image inserted in the middle of text? How do your make your note "stick"?
    • Note: And remember, websites are much more dynamic than "static" ebooks.
  • Different levels of annotations
    • From Private (only for you) -> Fully Public (everyone can see).

and a lot of other stuff.

Quote:
Originally Posted by DarkTrick View Post
I've never thought of that (probably as I don't let my ebooks being updated. But this might indeed be a hurdle.
When ebooks are updated on Amazon:
  • Amazon sends you a popup saying the publisher has updated the ebook + if you want to update your copy.
  • They "warn you" of losing all your highlights, etc.
  • You then have to explicitly accept it.

It's up to you to then balance the ebook fixes vs. losing your notes.

- - -

Side Note: I assume other stores do similar.

Kobo, as of a few years ago, locks you to the ebook-version-when-you-purchased, unless you contacted them directly.

I know back in 2019, when I informed Kobo one of the ebooks I purchased was accidentally set to "French", they updated my copy (and let me know the publisher already had it fixed within the months of my purchase + when I began reading).

Quote:
Originally Posted by DarkTrick View Post
I could imaging though, that a git-diff approach might solve this rather easily.
lol. Don't make me laugh. You probably haven't seen the underlying changes that occur when updating books OR doing a Calibre-convert.

There ain't nothing "easy" about git diff!

- - -

Side Note: For some of the super in-depth discussion, see this fantastic 5-part blog post in 2021: "Optimizing Git's Merge Machinery".

You wouldn't believe all the crazy edge-cases that come up.

- - -

But, if you read the "Open Annotation" specs, they do describe how you might try to keep comments attached, even if the underlying text/code is changing often.

(Again, think something like Microsoft Word / LibreOffice / Google Docs comments. They've already been trying to tackle similar problems for years.)

- - -

Complete Side Note: The latest insanity I ran across while digging through LibreOffice "Comment bugs"... did you know that Word allows you to COLOR text/backgrounds/highlights and carry over all this insane extra formatting within comments?

Not to mention:
  • nested replies
  • commenting in headers/footers
  • images-within-comments

... and all sorts of other insanity.

Quote:
Originally Posted by DarkTrick View Post
Anyway, I guess the conclusion for now is mostly (1) use books or (2) use a reader with a very easy notes-export functionality.
Yes, pretty much this. Find one tool/workflow that you like, test the export functionality, and stick with it.

One of my frustrations is, after export, most only carry the raw text of the highlight.

(Many times, I only highlight a handful of words—not entire sentences/paragraphs. I jump to the comment, then need to read the surrounding context.)

If you read the Open Annotate specs above, they also describe ways to export strings to the left/right of the highlight... to try to make it a little more robust.

Last edited by Tex2002ans; 08-08-2022 at 04:46 PM.
Tex2002ans is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Annotations in epub - Nova2 ottischwenk Onyx Boox 1 04-12-2020 01:39 PM
Boox Note/Max2: Question on making annotations to epub (and general handling of epub) johnnyb Onyx Boox 0 06-08-2018 06:59 AM
Transfer epub annotations BuzzB Kobo Reader 1 10-07-2014 09:10 PM
Touch support for annotations and bookmarks on epub? jfca283 Kobo Reader 8 11-03-2011 07:47 PM


All times are GMT -4. The time now is 07:22 AM.


MobileRead.com is a privately owned, operated and funded community.