iLiad Teasing :D

[rio]

Hi *,

Check the attached pic

I started to play a bit with the note taking feature of the iliad, which led me to wonder if it was possible to generate a pdf from the scribbles and the image used.. alas the java scribble merging application only seems to work for PDF scribble. I guess it would be trivial to modify the java app, but I had a look at the xml scribbling format, and when I saw it was so straightforward I quickly wrote myself a MacOSX viewer for the notes, using the png image as a background.

But then... I suddenly remembered the Ink handwriting recognition engine.

This thing comes straight from the ill-fated Apple Newton PDA (such a loss!), but what is nice is that it is available and installed by default on OSX

Turns out it's not too difficult to feed Ink a set of custom datapoints, and after some tweaking it doesn't work too bad apparently, as can be seen on the screenshot...

So far, this is only highly experimental code, and it'll probably take a bit more time to have a really usable application.

[daudi]

Very cool! This looks very promising. I don't suppose this is something that could be legally shared for use on non-Macs. I mean that ex-Newton code/application, is it?

[rio]

Originally Posted by daudi

Very cool! This looks very promising. I don't suppose this is something that could be legally shared for use on non-Macs. I mean that ex-Newton code/application, is it?

No, it's a system library :-/
For windows, I don't know if the stuff they use on tabletpc is downloadable for free for non-tabletpc and if it's easy to hook something like that (probably is). For linux I don't know a good HWR lib but one may exist.

If HWR engine are available, the app could be ported easily via GNUstep.

[Redfox]

That is way cool! When I got my iLiad and found out that HWR wasn't supported on the Mac I was very disappointed, but this could be the answer

If you need any beta testers, email me via the forum - I have some basic Unix knowledge, though I'm a relative newcomer to the Mac so I don't know diddly about OSX-specific scripting tools

[mrdini]

Originally Posted by rio

Hi *,

Check the attached pic

I started to play a bit with the note taking feature of the iliad, which led me to wonder if it was possible to generate a pdf from the scribbles and the image used.. alas the java scribble merging application only seems to work for PDF scribble. I guess it would be trivial to modify the java app, but I had a look at the xml scribbling format, and when I saw it was so straightforward I quickly wrote myself a MacOSX viewer for the notes, using the png image as a background.

But then... I suddenly remembered the Ink handwriting recognition engine.

This thing comes straight from the ill-fated Apple Newton PDA (such a loss!), but what is nice is that it is available and installed by default on OSX

Turns out it's not too difficult to feed Ink a set of custom datapoints, and after some tweaking it doesn't work too bad apparently, as can be seen on the screenshot...

So far, this is only highly experimental code, and it'll probably take a bit more time to have a really usable application.

Quite neat! I'm interested in this as well, as my V2 also has a touchscreen (& generates notes in .png format)...

How do you hook into the library...? (I had a browse through the ADC notes, but as far as I can tell, it only works with tablets...?)

[rio]

Originally Posted by mrdini

Quite neat! I'm interested in this as well, as my V2 also has a touchscreen (& generates notes in .png format)...

How do you hook into the library...? (I had a browse through the ADC notes, but as far as I can tell, it only works with tablets...?)

You can tell the framework that you will feed it points; so I simply "replay" the points from the .irx. The Ink engine will automatically send you the recognized text as carbon events, so you have to register for those and that's about it !

The only problem is that you are supposed to tell the engine when phrases are complete, and that seems to impact quite a bit the recognition. So far I just have a very crude way of segmenting the strokes in phrases, which kinda work but it shouldn't be difficult to come up with a vertical frequency method to classify the strokes, and that should be more robust.

[joblack]

Sometimes even I couldn´t read my own handwriting - how should a software do it ?

[rio]

Originally Posted by Redfox

That is way cool! When I got my iLiad and found out that HWR wasn't supported on the Mac I was very disappointed, but this could be the answer

If you need any beta testers, email me via the forum - I have some basic Unix knowledge, though I'm a relative newcomer to the Mac so I don't know diddly about OSX-specific scripting tools

I'll be quite busy in the next couple of weeks but should have more time after that to work on a usable application..

[mrdini]

Originally Posted by rio

You can tell the framework that you will feed it points; so I simply "replay" the points from the .irx. The Ink engine will automatically send you the recognized text as carbon events, so you have to register for those and that's about it !

The only problem is that you are supposed to tell the engine when phrases are complete, and that seems to impact quite a bit the recognition. So far I just have a very crude way of segmenting the strokes in phrases, which kinda work but it shouldn't be difficult to come up with a vertical frequency method to classify the strokes, and that should be more robust.

*Reads ADC* Ah, now I get it. Quite straightforward really!

Unfortunately, if I understand you, & the Ink Services docs correctly, this would only work with actual "vector"/points data i.e. a PNG image file wouldn't work as an input. Damn.

Hopefully, one day, I'll be able to hack the V2, & be able to obtain raw data from the touchpad...

Ah well, wish you all the luck with this!

[rio]

Originally Posted by mrdini

*Reads ADC* Ah, now I get it. Quite straightforward really!

Unfortunately, if I understand you, & the Ink Services docs correctly, this would only work with actual "vector"/points data i.e. a PNG image file wouldn't work as an input. Damn.

Hopefully, one day, I'll be able to hack the V2, & be able to obtain raw data from the touchpad...

Ah well, wish you all the luck with this!

I'm not sure I understand your point about the V2 ? Here's what I do: I have a simple script on the iliad that creates me a new "notepad" -- a simple folder with a manifest indicating that we're using a specific background PNG image (the gray horizontal lines shown on my screenshot). When I open this newly generated notepad I can scribble on it, and the scribbles are saved in the newly created "notepad" folder/document as a .irx file.

This irx file is a simple xml-encoded file containing the raw data from the scribbles. What I do then is I get the file on the mac, parse it to extract the data points, and use those along with the background png image to re-render the note in a cocoa view (which then allows me to print it or generate a pdf of it).

The recognition phase simply consist in feeding the same data points that to the Ink HWR engine, and I get back the recognized text. An additional thing to do is to segment the points in phrases, and so far my method is really crude and I want to implement a better segmenting tool.

So to sum it up, I don't see anything here that wouldn't work on the V2
(particularly considering I'm using the same OS version!)

edit: hum, maybe I just completely misunderstood you and the V2 is some kind of other tablet-like hardware, not an Iliad V2, that would exports notes in PNG... then yes if that's the case, it won't be easy. Although you can try to vectorise the PNG (should work quite well) and use the vectorised points to feed the Ink engine. But that's certainly a lot more work!

[mrdini]

Originally Posted by rio

So to sum it up, I don't see anything here that wouldn't work on the V2
(particularly considering I'm using the same OS version!)

Hanlin V2! :P Not iRex iLiad V2!

As far as I can tell, the notes are simply stored as .png files. No associated metadata/raw data.

[rio]

Originally Posted by mrdini

Hanlin V2! :P Not iRex iLiad V2!

Yup

Originally Posted by mrdini

As far as I can tell, the notes are simply stored as .png files. No associated metadata/raw data.

:-/ that makes things much harder indeed. One way would be to first vectorise the PNG (there's nice libs for that, iirc autotrace), then segment the points, then feed that back to a HWR engine.

[Redfox]

Originally Posted by rio

I'll be quite busy in the next couple of weeks but should have more time after that to work on a usable application..

No worries - it's not something I need to do urgently, just a nice possibility for the future

[Pode]

About the Windows side of things : there's also the (MS) Ink from the TabletPC architecture available on MS systems, as said earlier. Users of Vista should have the component installed by default. Windows XP (-non TPC version) can download the SDK/extension from the MS site : http://www.microsoft.com/downloads/d...displaylang=en

[rio]

Originally Posted by Pode

About the Windows side of things : there's also the (MS) Ink from the TabletPC architecture available on MS systems, as said earlier. Users of Vista should have the component installed by default. Windows XP (-non TPC version) can download the SDK/extension from the MS site : http://www.microsoft.com/downloads/d...displaylang=en

Sounds to be what we need yes (I'd have been surprised if it wasn't accessible). Had a very quick look at the classes and there's a stroke one, and you can save ink data. So it should be doable.