Exporting annotations, bookmarks and reading progress to the cloud

[alva]

i started to use the script, but i get the entries always twice

E.g.

### 📄 When You Smile, Melissa Brayden

*2024-07-20 14:46:25*\
OEBPS/Text/Prologue.xhtml

> “Don’t worry about what might happen until it does. You just wind up wasting a lot of energy that you could have spent”


### 📄 When You Smile, Melissa Brayden

*2024-07-20 14:46:25*\
Prologue

> “Don’t worry about what might happen until it does. You just wind up wasting a lot of energy that you could have spent”

Is there a possibility bto upgrade the script to stop nit from happening?

I have testet it with 3 different books and it was the same.

Thanks for providing the script it is very helpful to me

[Cyril Gabard]

Hello,

I changed the bookmarks script so it dont show multiples entries for the same book. I wanted to get the last bookmark and only the last. Try this.

Hope it will help.

Originally Posted by alva

i started to use the script, but i get the entries always twice

E.g.

### 📄 When You Smile, Melissa Brayden

*2024-07-20 14:46:25*\
OEBPS/Text/Prologue.xhtml

> “Don’t worry about what might happen until it does. You just wind up wasting a lot of energy that you could have spent”


### 📄 When You Smile, Melissa Brayden


*2024-07-20 14:46:25*\
Prologue

> “Don’t worry about what might happen until it does. You just wind up wasting a lot of energy that you could have spent”

Is there a possibility bto upgrade the script to stop nit from happening?

I have testet it with 3 different books and it was the same.

Thanks for providing the script it is very helpful to me

[Paperstarwriters]

Hello! thank you so much for this tool but unfortunately I can't seem to get any of my Highlights or Annotations. I keep getting a blank Markdown file that only reads as "# Kobo Notes" the date and time, and then "## Highlights" and nothing after that.

While It still did occur while using the exact command offered for NickleMenu, I did cut out the wifi part so I'm not sure if I need to replace it with something else?

I'm also viewing and ajusting the files on a macbook if that changes anything, but I've turned off the feature to auto expand all the files I open so I shouldn't have erased anything but I'm not sure.

[febalci]

Thanks to @qkqw and @Cyril, i added their corrections to the script.

Below is the version which prevents creating double annotations for chapter and filename.

Also, it has a Bookmarks part which shows all the page curls on the books.


Now i wonder is there a way to also upload the exported annotations .md file to cloud like gdrive or onedrive, or better an smb share on local network?

[febalci]

I managed to upload the annotations.md file to dropbox account or a php web server directly from kobo itself. Now once i understand how to create KoboRoot.tgz i will pack them all and post it here.

[febalci]

Couldn't had the will to handle under KoboRoot.tgz, so somebody else might do it maybe...

This script creates an .md file of annotations and bookmarks and copy the file to your dropbox account when you select "Sync Annotations" from nickel menu.


1. From https://github.com/fsantini/KoboClou...ocal/kobocloud download "curl" and "ca-bundle.crt" and copy it under .adds/notes folder on Kobo

2. Go to https://www.dropbox.com/developers/apps
Click “Create app”
Choose:
Scoped access
Select App Folder
Give your app a name (Should be unique worldwide)
Under OAuth 2, click “Generate access token”
Copy and use that token as DROPBOX_ACCESS_TOKEN in the script.

3. notes.sh:

#!/bin/sh
'
🧭 Steps
    Go to https://www.dropbox.com/developers/apps
    Click “Create app”
    Choose:
        Scoped access
        Select App Folder
    Give your app a name (Should be unique worldwide)
    Under OAuth 2, click “Generate access token”
    Use that token in your curl upload

From https://github.com/fsantini/KoboCloud/tree/master/src/usr/local/kobocloud
download curl and ca-bundle.crt to notes folder
'
DROPBOX_ACCESS_TOKEN="YOUR_DROPBOX_ACCESS_TOKEN"

NOTES="/mnt/onboard/.adds/notes"
# Note: Depending on your language this folder may have a different name
EXPORT_FOLDER="/mnt/onboard/Exported Annotations"
EXPORT_FILE="notes-$(date "+%Y-%m-%d").md"
EXPORT="$EXPORT_FOLDER/$EXPORT_FILE"
KEEP=21

DB="/mnt/onboard/.kobo/KoboReader.sqlite"
SQLITE="${NOTES}/sqlite3"

LD_LIBRARY_PATH="${NOTES}/lib:${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH

mkdir -p "$(dirname "$EXPORT")"

echo -e "# Kobo Notes\n" > $EXPORT
echo -e "*$(date -R)*\n" >> $EXPORT
echo -e "## Highlights\n" >> $EXPORT

# UTF-8 char table (decimal):
#      9: Tab
#     10: Line feed
#     32: Space
#     58: :
#     62: >
#     42: *
#     92: \
#   8230: …
#   9999: ✏
# 128278: 🔖
# 128196: 📄

SQL="SELECT TRIM(
  '### ' ||
  CASE
    WHEN b.Type = 'dogear' THEN
      char(128278, 32)
    WHEN b.Type = 'note' THEN
      char(9999, 32)
    WHEN b.Type = 'highlight' THEN
      char(128196, 32)
  END
  || c.Title || ', ' || COALESCE(c.Attribution, 'N/A') || char(10, 10, 42)
  || datetime(b.dateCreated) || char(42, 92, 10) /* force Markdown newline */
  || COALESCE(c1.Title, '') || char(10, 10, 62, 32) ||
  CASE
    WHEN b.Type = 'dogear' THEN
      COALESCE(ContextString, 'No context available') || char(8230, 10) /* only kepubs have context */
    ELSE
      REPLACE(              /* start Markdown quote */
        REPLACE(
          TRIM(             /* trim newlines */
            TRIM(           /* trim tabs */
              TRIM(b.Text), /* trim spaces */
              char(9)
            ),
            char(10)
          ),
          char(9), ''
        ),
        char(10), char(10, 62, 32, 10, 62, 32)) /* continue Markdown quote for multiple paragraphs */
      || char(10, 10)
      || COALESCE(b.Annotation, '') || char(10)
  END, char(10)
  ) || char(10, 10)
  FROM Bookmark b
    INNER JOIN content c ON b.VolumeID = c.ContentID
    LEFT OUTER JOIN content c1 ON (c1.ContentId LIKE b.ContentId || '%' AND c1.ContentType != 9)
  ORDER BY c.Title ASC,
           c.VolumeIndex ASC,
           b.ChapterProgress ASC,
           b.DateCreated ASC;"

$SQLITE "$DB" "$SQL" >> $EXPORT

echo -e "## Bookmarks:\n" >> $EXPORT

SQL="SELECT TRIM(
  '### ' ||
  CASE
    WHEN b.Type = 'dogear' THEN
      char(128278, 32)
  END
  || c.Title || ', ' || COALESCE(c.Attribution, 'N/A') || ', ' || c.Publisher || ', ' || strftime('%Y', date(datetime(c.dateCreated))) || char(10, 10)
  || strftime('%d/%m/%Y', date(datetime(b.dateCreated))) || ' ' || time(time(datetime(b.dateCreated)), '+1 hours') || char(10, 58, 10) /* force Markdown newline */
  || COALESCE(c1.Title, 'No chapter title') ||', ' || c.___PercentRead || '%  read' || char(10, 10, 62, 32) ||
  CASE
    WHEN b.Type = 'dogear' THEN
      COALESCE(ContextString, 'No context available') || char(8230, 10) /* only kepubs have context */
    ELSE
      REPLACE(              /* start Markdown quote */
        REPLACE(
          TRIM(             /* trim newlines */
            TRIM(           /* trim tabs */
              TRIM(b.Text), /* trim spaces */
              char(9)
            ),
            char(10)
          ),
          char(9), ''
        ),
        char(10), char(10, 62, 32, 10, 62, 32)) /* continue Markdown quote for multiple paragraphs */
      || char(10, 10)
      || COALESCE(b.Annotation, '') || char(10)
  END, char(10)
  ) || char(10)
  FROM Bookmark b
    INNER JOIN Event e ON e.ContentId = b.volumeId
        INNER JOIN content c ON b.VolumeID = c.ContentID
    LEFT OUTER JOIN content c1 ON (c1.ContentId LIKE b.ContentId || '%' AND c1.ContentType != 9)
  WHERE EventType=8 AND b.Hidden='false' AND b.ContextString!=''
  GROUP BY b.dateCreated, c.BookTitle
  ORDER BY c.BookTitle ASC, b.dateCreated DESC;"

$SQLITE "$DB" "$SQL" >> $EXPORT

echo -e "## Currently reading:\n" >> $EXPORT

SQL="SELECT
  '- ' || c.Title || COALESCE(', ' || c.Attribution, '')
  || ' (' || COALESCE(c1.Title || ', ', '') || c.___PercentRead || '% read' || ')'
  FROM Content c
  LEFT OUTER JOIN Content c1 ON (
    c.ContentID = c1.BookID
    AND c1.ContentType = 899
    AND REPLACE(c1.ContentID, '!', '/') LIKE /* get chapter id without anchor or query string */
      '%' || SUBSTR(c.ChapterIDBookmarked, 1, INSTR(c.ChapterIDBookmarked, '#') + INSTR(c.ChapterIDBookmarked, '?') - 1) || '%'
  )
  WHERE c.ContentType = 6
    AND c.ReadStatus = 1
    AND c.IsDownloaded = 'true'
  ORDER BY c.___PercentRead DESC,
           c.Title ASC,
           c.Attribution ASC;"

$SQLITE "$DB" "$SQL" >> $EXPORT

# Clean up old notes
cd "$EXPORT_FOLDER"
for i in $(ls -v notes* | head -n -$KEEP); do
  rm "$i"
done


'
# PHP Server:

- create "uploads" folder in www root folder with chown 777
- create "upload.php" file in www root folder

## PHP:
<?php
if ($_SERVER['REQUEST_METHOD'] === 'POST' && isset($_FILES['file'])) {
    $uploadDir = __DIR__ . '/uploads/';
    $filename = basename($_FILES['file']['name']);
    $targetPath = $uploadDir . $filename;

    // 🔒 Restrict allowed file extensions
    $allowedExtensions = ['txt', 'md'];
    $extension = strtolower(pathinfo($filename, PATHINFO_EXTENSION));

    if (!in_array($extension, $allowedExtensions)) {
        http_response_code(400);
        echo "Error: Only .txt and .md files are allowed.";
        exit;
    }

    // Proceed with upload
    if (move_uploaded_file($_FILES['file']['tmp_name'], $targetPath)) {
        echo "File uploaded successfully: " . htmlspecialchars($filename);
    } else {
        echo "Upload failed.";
    }
} else {
    echo "No file uploaded.";
}
?>

## command in this script file:
/mnt/onboard/.adds/notes/curl -X POST -F "file=@$EXPORT" http://server-ip/upload.php
'

/mnt/onboard/.adds/notes/curl --cacert /mnt/onboard/.adds/notes/ca-bundle.crt -X POST https://content.dropboxapi.com/2/files/upload \
    --header "Authorization: Bearer $DROPBOX_ACCESS_TOKEN" \
    --header "Dropbox-API-Arg: {\"path\": \"/$EXPORT_FILE\",\"mode\": \"add\",\"autorename\": true,\"mute\": false}" \
    --header "Content-Type: application/octet-stream" \
    --data-binary @"$EXPORT"

[febalci]

or...

Unzip this file and copy all to /mnt/onboard/.adds/notes/ folder.(You might want to copy your Dropbox access token to notes.sh beforehand)

Do Step 2 from the previous mail, creating a Dropbox app and get Access token.

Copy the access token to <YOUR DROPBOX ACCESS TOKEN> part in the notes.sh.

Create your nickelmenu 'Sync Annotations' item (From message 1 in this thread)

[Cyril Gabard]

Hello @febalci,
I tried the second method but didn't manage to export to my Dropbox account. I could see the Exported annotations folder on my Kobo but couldn't see any md file in my Dropbox despite the fact that i followed every steps (Dropbox apps, Token, paste token.
Could you be more didactic, please ?

[febalci]

Sorry Cyril for the late reply. Forgot my Kobo on the plane, took me a while to get it back

I found out the errors in my previous post. Multi line comments and Dropbox short lived access tokens. Here are the renewed steps from scratch:

Use this new zip file notes20250521.zip attached in this message.


🔧 Step 1: Create a Dropbox App

• Go to: Dropbox App Console
• Click "Create App"
• Choose: Scoped access and App Folder
• Set permissions: Enable these: files.content.write and files.content.read
• ✅ Save App Key and App Secret for the script!

🔑 Step 2: Get a Refresh Token (ONE-TIME manual setup)
You need to log in and approve access.

• Visit this URL in your computer browser, with your App Key:
  

  https://www.dropbox.com/oauth2/authorize?client_id=YOUR_APP_KEY&response_type=code&token_access_type=offline

• Log in and allow access
• It redirects to a page 'ACCESS CODE GENERATED'. Copy the code.
• Now exchange it for a refresh token using this command:
  

  curl -X POST https://api.dropboxapi.com/oauth2/token \
    -u YOUR_APP_KEY:YOUR_APP_SECRET \
    -d code=ACCESS_CODE_GENERATED \
    -d grant_type=authorization_code

  * If you don't have curl on your computer, you can use the curl in notes folder by telnetting to your Kobo device.
• The output will include:
  

   "refresh_token": "YOUR_REFRESH_TOKEN_HERE"

• ✅ Save this refresh_token for the script!

Sorry for being so complicated but dropbox is a headache. Hopefully it will work this time; mine works succesfully.

[Cyril Gabard]

Thanks @febalci, i will try later and tell you. Did you try with Google ?

[Cyril Gabard]

Oups, here is what i get after the "Curl call" :

<!DOCTYPE html>
<html>
<head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Dropbox - 404</title>
<link href="https://cfl.dropboxstatic.com/static/metaserver/static/css/error.css" rel="stylesheet" type="text/css"/>
<link rel="shortcut icon" href="https://cfl.dropboxstatic.com/static/images/favicon.ico"/>
</head>
<body>
<div class="figure">
<img src="https://assets.dropbox.com/www/en-us/illustrations/spot/look-magnifying-glass.svg" alt="Error: 404"/>
</div>
<div id="errorbox">
<div class="not-found"> <h1>Error (404)</h1> We can't find the page you're looking for. <div class="not-found--links"> Here are a few links that may be helpful: <ul> <li><a href="https://www.dropbox.com/home?_tk=fof">Home</a></li> <li><a href="https://www.dropbox.com/help?_tk=fof">Help center</a></li> <li><a href="https://www.dropbox.com/login?_tk=fof">Sign in</a></li> <li><a href="https://www.dropbox.com/register?_tk=fof">Get a free account</a></li> <li><a href="https://www.dropbox.com/plus?_tk=fof">Dropbox Plus</a></li> <li><a href="https://www.dropbox.com/business?_tk=fof">Dropbox Business</a></li> </ul> </div> </div>
</div>

</body>
</html>
curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535
curl: (3) URL rejected: Bad hostname
curl: (3) URL rejected: Bad hostname

[febalci]

First of all, don't forget to unzip and install files in notes20250521.zip file from the previous message attachment, into the notes folder

If you are using the Kobo Telnet curl, the command on Kobo telnet under notes folder should be as follows to get the refresh code:

./curl --cacert ./ca-bundle.crt -X POST https://api.dropboxapi.com/oauth2/token \
  -u YOUR_APP_KEY:YOUR_APP_SECRET \
  -d code=<ACCESS_CODE_GENERATED> \
  -d grant_type=authorization_code

Access codes generated from the web page are one time use only, and time restricted. Once you get the Refresh Code, then you can use it any time.

[Cyril Gabard]

Sorry to bother. I dont know how to use the Kobo Telnet curl, i was trying from my PC