PRS-T1 PRS-T1 Annotations Exporter

[Yoths]

gumalcan
glukkon
The errors seem to be thrown by the XML parser. Could you provide me with the books which make problems, so I can debug the parsing process?

glukkon
To remove the char limit for PDF files, I first have to learn how to extract text from them and how to decrypt the text position descriptions. Any information about that is welcome...

enitro
May I ask what do you want to expand?

[martz]

This application is really-really great news. Thank you Yoths!

Struggling with a book deadline, I visited this forum after a while in hope for exactly these news. And -- a minute later I also found out about this excellent Calibre plugin as well. So, a really good day for me, and how I appreciate this community!

However -- one tiny "please": could you add an option for exporting (text) notes in plain text instead of HTML? Something like this, separated by a blank line and with the page number in brackets:

Annotation text annotation text annotation text annotation text. (pp 83)

Next annotation next annotation next annotation. (pp 84)

For explanation: I am using this awesome Perl script for generating tag clouds from my notes. And, of course, I could clean up your HTML file the corrseponding way by myself (with a macro) but I'd save a few keystrokes if I didn't need to do that :-)

Thanks for any feedback.

(And, BTW, this Perl tagcloud is really great; it has proven itself as an excellent "last step" in my PRS-T1 reading workflow. Check it out everybody.)

[enitro]

Thanks Yoths, for the reply.

I just want to look around the code and take requests, expand functionality and so on, you can PM me the source.

Thanks.

[papagelos]

I would like to do this:

1. Read a book, highlighting important parts.
2. Use this utility to gather all my hightlights into 1 document.
3. Import that document into my reader so I can study only the important parts.

Any chance of you beeing able to code this? I have not been able to import back to the reader your output. It is not in .epub or other acceptable format for it.

Or if you know any other means to accomplish the 3 steps, let me know.

EDIT: Also, If I marked 220 characters, and extended with 420 characters, that means there are 200 characters in the exported text that are NOT HIGHLIGHTED. Perhaps your util can now detect those, and keep only the highlighted text and make the .epub note-book only out of those?

[Yoths]

A new version is available (links in the first post are updated).

What's new:
- customizable HTML export (via "HtmlExportStyles.xml")
- marginal bugfixes

enitro
You can download the source code from the first post. I'm sorry I kept you waiting...

martz, papagelos
I'm not yet going to implement other export formats. As workaround you can design your own HTML export style and convert the HTML output to whatever you want with Calibre.

[enitro]

Thanks Yoths, appreciate it.

[martz]

Originally Posted by Yoths

martz, papagelos
I'm not yet going to implement other export formats. As workaround you can design your own HTML export style and convert the HTML output to whatever you want with Calibre.

I'll do that, thanks very much for the tip.

[ebiel]

I just want to thank you for the very useful program! It does exactly what I was looking for

Cheers
Ebiel

[Toxaris]

I got a PRS-T1 yesterday and I think that the highlight would be a handy thing for error hunting. To test it, I highlighted a few words and hooked the PRS-T1 to my computer. I fired up your program and it found that the book that I made a highlight in. When I press the export button, the following error pops up:

Any ideas? The ePUB validates and is not restricted in any way.

[Yoths]

Toxaris
Would you please post the the error message as well?

[Toxaris]

The picture is the error message...

[Yoths]

Hmm... I don't see any picture in your post...

edit
OK, I got the picture (I have no access to dropbox here at work, so it is invisible for me in your post...)

Toxaris
Could you please try this version?

[Toxaris]

That version works without an issue!

[Yoths]

A new version is available (links in the first post are updated).

What's new:
- bugfix: export annotations containing "<", ">" and "&" charachters
- bugfix: parse XML files importing a local DTD

[mhr]

Originally Posted by Yoths

I wrote a small tool for exporting annotations from the T1 to HTML.
...

Inspired from Your work and after looking into the source code I wanted to have something similar for pure '*.txt' files, to be able to mark errors in such pure text files without any text formatting on the Sony PRS T1 and to correct them later on the PC. I looked into the sq-database and recognized, how Sony saved the begin and end of the marks (its a binary coding) for '*.txt' files. Then I wrote a very basic Python script (no error handling) to augment the naked .txt files with marker symbols, which can easily be searched for in a standard text editor (I choose [[[ ]]] for the markers in the script).

To use the script, copy the 'books.db' file from the Sony reader (of course after the highlighting annotations are made with the Sony) in Your working directory, make two subfolders called joined and marked, copy all text files from Sony reader, which should be augmented by the Sony highlighting annotations into the subfolder joined and call the script. The augmented text files with the same file name are created then in subfolder marked and can be treated by Your favorite text editor on the PC. Here is the script:

#!/usr/bin/python
# insert_text_markers
# Reads 'books.db' in current directory and converts
# each TXT-file in subdir 'joined' into TXT-file in subdir 'marked'
# with marked text indicated in 'books.db' marked by '[[[' and ']]]'.

import os,sys
import sqlite3

book2ind   = {}  # Text file names (without path) to contents index.
annotation = []  # Array of annotations with rows (cid,txt,ca,cb).

# Calculates byte offset from position coding in annotation table:
def pos2ind(pos):
  if pos[0:4] == "TXTk":
    ind = 0
    for k in range(15,11,-1):  ind = 256*ind + ord(pos[k])
    return ind
  else:  return None

# Compare function to sort annotations by range:
def cmp_pos(a,b):
  if a[2] < b[2]:  return -1
  if a[2] > b[2]:  return  1
  if a[3] < b[3]:  return -1
  if a[3] > b[3]:  return  1
  return 0

# Reads book names and annotation ranges:
def readdb(name):
  conn = sqlite3.connect(name)
  c    = conn.cursor()
  for row in c.execute("SELECT * FROM books"):  book2ind[row[13]] = row[0]
  for row in c.execute("SELECT * FROM annotation"):
    cid = int(row[1])      # The content id of the corresponding textfile.
    txt = row[6]           # The marked text.
    ca  = pos2ind(row[7])  # Start index of marked text in text file.
    cb  = pos2ind(row[8])  # End   index of marked text in text file.
    if ca != None and cb != None:
      annotation.append((cid,txt,ca,cb))
  conn.close()

def add_marks(all_txt,cur_cid):
  k   = 0
  n   = len(all_txt)
  arr = []
  for (cid,txt,ca,cb) in annotation:
    # print(cid,ca,cb,txt)
    if k <= ca:
      arr.append(all_txt[k:ca])
      arr.append("[[[")
      arr.append(all_txt[ca:cb])
      arr.append("]]]")
      k = cb
  arr.append(all_txt[k:n])
  return "".join(arr)

readdb("books.db")
annotation.sort(cmp_pos)

for name in os.listdir("joined"):
  if name in book2ind:
    print("Add marks to '%s' ..." % name)
    fh  = open(os.path.join("joined",name),"rb")
    content = fh.read()
    fh.close()
    content = add_marks(content,book2ind[name])
    fh = open(os.path.join("marked",name),"wb")
    fh.write(content)
    fh.close()

The code runs fine under Linux with python and python-sqlite3 installed.
Probable it should also run under Windows. I don't need a more sophisticated setup and changes for Your needs should be manageable.