Progress-Synced Reading Soundtracks

[apopagasm]

Hi folks,

I've just signed up here and in fact just learned that there's such a thing as Kindle jailbreaking/customizing, but I've had an idea that I think would be extremely cool to anyone who enjoys reading while listening to music, & am interested in if this would be at all possible to implement (for the record I do UX and visual design for iOS apps, no coding at all).

Idea is this: A "soundtrack" created specifically and manually for a book, that consists of a series of playlists for different settings or actions (e.g. fight scenes), etc. These playlists are live-synced to your progress in a book, so as you read you'll hear music specifically selected for the scene, setting, etc that you're currently reading. Playlists probably shuffle, and the transition from one to another is probably a smooth long crossfade. I'm thinking playlists are assembled from various movie/game/etc soundtracks, licensing issues aside.

Would involve a (crowd-sourced?) "book map" for each compatible book tying playlists to certain stretches of the book, then the playlists could get as specific as wanted; could have a generic fantasy fight scene playlist to cover a large # of books or one specifically created for one scene in a book (e.g. LOTR fight scene music from the movie OST, for the same fight scene in the book).

So as you read about characters exploring a mystical arabian (or whatever) city, stuff like this plays. Then they go get in a fight, arabian-sort of fight music plays, then they leave & travel thru a desert and stuff like this plays. And on and on for the whole book. You can listen to the Harry Potter movie soundtracks on shuffle while reading Harry Potter, but imagine how much better it would be if they were specifically ordered and curated based on where you are in the book.

Anybody else think this would be super cool, and anyone have any idea if this is at all doable? From some quick googling it seems there's no official API that allows access to reading progress, but perhaps there is a way if jailbroken.

[knc1]

There is no **official** API for anything gray-scale Kindle.
These devices run an OS implemented by Amazon themselves.

Your info. box lists "Voyage" as the device you own.
How are you going to listen to anything?
In case you haven't noticed, your Voyage does not have any speakers (or other support for audio media).

The last Kindle (grayscale) model that supported audio was the Kindle Touch (called: K5 here).

[Althelney]

Yeah... honestly that sounds like a project for almost any device other than a kindle. Maybe an android?

In any case, it sounds like a massive undertaking to annotate a functionally infinite number of books when you could just, you know, make a thematic playlist and change the track when appropriate.

[apopagasm]

Originally Posted by knc1

There is no **official** API for anything gray-scale Kindle.
These devices run an OS implemented by Amazon themselves.

Your info. box lists "Voyage" as the device you own.
How are you going to listen to anything?
In case you haven't noticed, your Voyage does not have any speakers (or other support for audio media).

The last Kindle (grayscale) model that supported audio was the Kindle Touch (called: K5 here).

The only need for the actual device (kindle) would be the progress%; I'm imagining a PC (or web) client to do the actual playing and house the audio files, etc.

Was talking to my boss (iOS developer) and he mentioned it'd be "a very interesting technical challenge to try to auto pick music by analyzing text" which is indeed interesting, though just a way for users to make their own maps & link them to their own playlists/audio files would be the place to start. But yeah I am just ideating (?) now cuz that's what I do, though I totally agree with "a lot of people would use something like this if it were really well executed".

This is definitely possible with a custom eReader app, i.e for iOS or Android, but I'm wanting to find out if it's possible to send out that progress% from a jailbroken kindle (any kindle not necessarily the one I have) cuz those are much better to read on than a regular tablet.

[NiLuJe]

Probably. Although forget about matching that with actual content on a DRM'ed book. Which potentially severely limits its use, given the current landscape.

I'd also point you to the multitudes of companies creating 'enhanced' eBooks (either as Apps or trying to leverage the mess^Wfine standard that is ePub 3), that might be a better way to achieve this kind of thing.

[knc1]

Originally Posted by apopagasm

- - - - -
I'm wanting to find out if it's possible to send out that progress% from a jailbroken kindle (any kindle not necessarily the one I have) cuz those are much better to read on than a regular tablet.

Ah, so, your actual question appears.

Short answer: Yes
Longer answer: So what?
Long answer:

I don't think the progress percent has a significantly stable relationship between its value and the current text position from Kindle instance to Kindle instance.

An added source of uncertainty -
Our e-reader add-ins support a very wide range of e-book formats.
I am not sure that even the formats where "byte counting" is used will be repeatable enough.

6900 - I need a life.

[apopagasm]

Originally Posted by knc1

Ah, so, your actual question appears.

Short answer: Yes
Longer answer: So what?
Long answer:

I don't think the progress percent has a significantly stable relationship between its value and the current text position from Kindle instance to Kindle instance.

An added source of uncertainty -
Our e-reader add-ins support a very wide range of e-book formats.
I am not sure that even the formats where "byte counting" is used will be repeatable enough.

I don't know what effects a book being DRM'd might have (@NiLuJe), and yeah certainly there are a bunch of issues but I haven't seen any so far that are major obstacles. If one can analyze the text to try to auto-pick music based on it, certainly one could analyze the text to sort of align one ebook (that the map was made from) onto another, even if one has chapter 1 starting at loc 200 and the other at loc 50. Unless an ebook is severely screwed up (i.e. a PDF with spaces between every character) any two different copies of one book will have the same sentences, yeah?

My actual question was also at the end of my first post by the way

edit: and "make a thematic playlist and change the track when appropriate" is far too much work to be realistic, imo; no one's gonna stop reading when they get to an action sequence or the setting changes to pick a new song. I do have great success listening to music while reading but I imagine this would be a whole new experience & be extremely nice to use if done well

[knc1]

Well, send us a link to your working prototype when you have one.

[NiLuJe]

@apopagasm: A location pointer in a DRM'd mobi7/mobi8 basically points to word-soup: encrypted content. Hence: useless.

And I'm not sure they're actually stable (i.e., if you by chance have a decrypted copy of the book, will the pointer from the DRM version actually point to the right place in the decrypted version (or vice versa)? Not quite sure, you'd have to check with the Mobi gurus in the appropriate subforum).

[badgoodDeb]

Different people read at different speeds. You would need a differing amount of the music then. Person A might need 10 minutes of "danger scene" music to cover the time (s)he's reading through that scene. Person B might need 20 minutes, if you see what I mean.

I, on the other hand, tend to focus on the music, and not be able to read while listening. I don't want any music while I'm reading! So I won't be a client, but that's okay.

[apopagasm]

Originally Posted by badgoodDeb

Different people read at different speeds. You would need a differing amount of the music then. Person A might need 10 minutes of "danger scene" music to cover the time (s)he's reading through that scene. Person B might need 20 minutes, if you see what I mean.

I, on the other hand, tend to focus on the music, and not be able to read while listening. I don't want any music while I'm reading! So I won't be a client, but that's okay.

It's not time-based at all, think like a video game. User can pause at one scene for 3 hours and just hear the playlist associated with it shuffle, or speed through it and not hear the entire playlist

I am optimistic of a working prototype fairly soon, also (edit: not so fast). Temp name & logo (or, permanent until a better one comes along)



(soundstripe being a synonym of "soundtrack")

[eschwartz]

Originally Posted by knc1

6900 - I need a life.

Heh. 7829.
I joined later...

[eschwartz]

Originally Posted by NiLuJe

@apopagasm: A location pointer in a DRM'd mobi7/mobi8 basically points to word-soup: encrypted content. Hence: useless.

And I'm not sure they're actually stable (i.e., if you by chance have a decrypted copy of the book, will the pointer from the DRM version actually point to the right place in the decrypted version (or vice versa)? Not quite sure, you'd have to check with the Mobi gurus in the appropriate subforum).

I believe everything is done by decrypting the book on the fly.

The pointer is calculated based on file offsets on the content passed to the actual book reader, whether that came directly from the book file, or was decrypted.