Is there really no local backup of Notes?

[dnquark]

Originally Posted by adrians

The original, i.e. non-backup, folder is non-accessible to apps other than notes, I guess, no? It would be nice to be able to periodically sync to OneNote.

You don't have direct access to db obviously but now that I think of it there _might_ be a way to access underlying data through the node model class in the SDK so that you don't actually need to reverse-engineer the file format. See https://github.com/onyx-intl/OnyxAnd...oActivity.java

[diabl0w]

Originally Posted by dnquark

I just grabbed the db files and poked around If you are curious, I suggest getting the backup zip file from local storage (it will be in `/sdcard/note/backup/local/`), and opening it with SQLite browser. Most of the fields are fairly self-explanatory with the exception of binary stroke blobs, but these are arrays of 24-byte structs, (4 floats and a long, big-endian, corresponding to x, y, pressure, size, timestamp). Size is always 0. It seems that for strokes, if the SVG I get from PDF has N points, there will be 2N points in the binary blob. The scaling/units in the float values isn't clear, so it's difficult to do side-by-side comparison with SVG output. But if you just do a scatterplot of the points, you get the strokes you expect.

That's about all I found out, if you are able to reverse-engineer more of the file format that would be pretty cool. It seems quite doable, but I don't really have the bandwidth for it now.

I am doing some poking around as well. Not sure if this is already known, but I can also confirm that local backup of notes is literally the same databases that are found in the /data/data/com.onyx.android.note/databases folder. Meaning that if you do a backup, mess with the databases, and rezip, then restore backup in the notes app, it will literally just copy paste the database files back in the Note app's database folder and your modified notes will be available in the app. Will do some more digging.

[diabl0w]

some more interesting info: notes that you take in the neoreader app, when split screen with a book and taking notes for the book, are also stored in the same database as your notes database. however, notes taken for books are obviously not visible in the notes app. This is controlled by a single value in the notes database file "ShapeDatabase.db". In the "NoteModel" table of the ShapeDatabase.db, you will find two columns that are null for notes app notes, but have a value for books app notes: associationId and associationType. The associationId is the unique id that onyx gives each of the books in your library. The associationType is just a value of '1' for book notes and '0' for regular notes. All that is needed to make a book note visible in the notes app is to toggle the associationType to 0. Doing this will disassociate the note from the book app, however, so you have to change it back to use it when taking notes with the book again. That being said, it is easy for us with root to quickly toggle the visibility of the note by using sqlite and just toggling the associationType value for rows who's column is not NULL for associationId column

[adrians]

Originally Posted by diabl0w

however, notes taken for books are obviously not visible in the notes app.

There's tool icon in the notes app (the third one from the top left, showing a pencil on top of a page with lines on it) which lets you switch to the notes taken for books.

[diabl0w]

Originally Posted by adrians

There's tool icon in the notes app (the third one from the top left, showing a pencil on top of a page with lines on it) which lets you switch to the notes taken for books.

oh wow... whoops! thanks for the tip, i had no idea that was there.
nonetheless my time was not wasted, as now I can also merge notes among other things.

[Vermilion Scans]

Is there a way to export and import highlights and scribbles from/to epub and pdf files? I'm trying to migrate my books from a Max2 to a MaxLumi, without loosing my annotations. Hope this is on topic, thanks

[BooxDude]

Originally Posted by diabl0w

I am doing some poking around as well. Not sure if this is already known, but I can also confirm that local backup of notes is literally the same databases that are found in the /data/data/com.onyx.android.note/databases folder. Meaning that if you do a backup, mess with the databases, and rezip, then restore backup in the notes app, it will literally just copy paste the database files back in the Note app's database folder and your modified notes will be available in the app. Will do some more digging.

Hey Guys, I'm trying to merge 2 DBs from 2 different devices to restore on a 3rd.
Related Questions:
1) For accessing the /data/data/.../databases folder -- did you root your device to access this? b/c when I hit it from a Droidvim terminal, I do not have read or any permissions
2) For the those who are modifying the files offline, then rezipping, I've been having trouble restoring from zips where I've 1) unzipped 2) made a slight mode to the shapes db (e.g. delete an entry for a deleted folder) and 3) rezipped. Do y'all have any tips here? I've used OSX's native compression from the UI as well as the zip from bash.

Thanks!

[diabl0w]

Originally Posted by BooxDude

Hey Guys, I'm trying to merge 2 DBs from 2 different devices to restore on a 3rd.
Related Questions:
1) For accessing the /data/data/.../databases folder -- did you root your device to access this? b/c when I hit it from a Droidvim terminal, I do not have read or any permissions
2) For the those who are modifying the files offline, then rezipping, I've been having trouble restoring from zips where I've 1) unzipped 2) made a slight mode to the shapes db (e.g. delete an entry for a deleted folder) and 3) rezipped. Do y'all have any tips here? I've used OSX's native compression from the UI as well as the zip from bash.

Thanks!

1) yes
2) Deleting probably wouldnt work, because when it restores the backup I am guessing it is only additive. Meaning, if i have 2 notes and then create a backup and then create a third note in the app and then restore the 2 previous notes, its not gonna delete my 3rd new note when restoring the 2 others, which is essentially what it seems like your trying to have happen. Sorry for being misleading in my previous post

[BooxDude]

Managed to solve my own issue. Bottom line for recompressing your backup, don't use zip on OS X, use something on Linux.

I had success with: Zip 3.0 compiled with GCC 6.3.0 for Linux ELF.

Two bonus questions:
1) since this is one of the technically deeper threads I've seen for Boox, do you all have any other favorite threads that are doing something fairly custom / deep on them?
2) I noticed I can do offline OCR on notes on my old Note Pro, where as the new Notes all require internet connectivity. Has anyone seen anyway to mess with that setting or seen anything related?

Thanks and happy Booxing!

[amd]

Originally Posted by dnquark

I just grabbed the db files and poked around If you are curious, I suggest getting the backup zip file from local storage (it will be in `/sdcard/note/backup/local/`), and opening it with SQLite browser. Most of the fields are fairly self-explanatory with the exception of binary stroke blobs, but these are arrays of 24-byte structs, (4 floats and a long, big-endian, corresponding to x, y, pressure, size, timestamp). Size is always 0. It seems that for strokes, if the SVG I get from PDF has N points, there will be 2N points in the binary blob. The scaling/units in the float values isn't clear, so it's difficult to do side-by-side comparison with SVG output. But if you just do a scatterplot of the points, you get the strokes you expect.

That's about all I found out, if you are able to reverse-engineer more of the file format that would be pretty cool. It seems quite doable, but I don't really have the bandwidth for it now.

It has been quite some time since the last post, however if someone is still interested, I have had a look at the format as well, thanks for the pointers
Here is a simple python script I threw together. It parses the sqlite db's and the binary blobs as well and renders the result to a skia canvas that can be saved to a png. I also extracted the keywords that are stored in the sqlite database (the Boox Note App does handwriting recognition quite well) and you can also pass a search term to the python script and it will draw a red rectangle around the recognized word if one is found.
Here is the code:
https://github.com/DAmesberger/boox-notes-backup-parser

I am now considering porting this to a proper implementation in Rust to convert this into SVG and attach it to Joplin Notes.