Audio (video) - Kindle. Getting it up and running, Code level.

[geekmaster]

I just repeated the above experiment on my K5, using top instead of htop. It reported that gmplay used 15% CPU (as expected because the K5 sleeps while the K3 is inside the eink update call). The tones program immediately aborts with the following errors:

ALSA lib pcm_hw.c:1429:(_snd_pcm_hw_open) Unknown field hint
Playback open error: Invalid argument

It appears that the K3 eink update calls are the culprit, consuming all that CPU time during system calls (which was also reported by hawhill some time ago). There must be some nasty wait loops or spinlocks, as mentioned before.

Those long busy eink updates on a K3 may interfere with sound, unless the sound is interrupt driven.

At least sound and video should work well together on a K4 or newer (after porting the sound code to the newer kindles).

FYI, the K3 spends about 140 msec full frame eink update calls, so sound buffering should be at least that long, and preferably synchronized to the framerate. On a DX and DXG, the 600x800 (not fullscreen) eink updates take about 650 msec to compete.

[twobob]

Originally Posted by geekmaster

I just repeated the above experiment on my K5, using top instead of htop. It reported that gmplay used 15% CPU (as expected because the K5 sleeps while the K3 is inside the eink update call). The tones program immediately aborts with the following errors:

ALSA lib pcm_hw.c:1429:(_snd_pcm_hw_open) Unknown field hint
Playback open error: Invalid argument

It appears that the K3 eink update calls are the culprit, consuming all that CPU time during system calls (which was also reported by hawhill some time ago). There must be some nasty wait loops or spinlocks, as mentioned before.

Those long busy eink updates on a K3 may interfere with sound, unless the sound is interrupt driven.

At least sound and video should work well together on a K4 or newer (after porting the sound code to the newer kindles).

FYI, the K3 spends about 140 msec full frame eink update calls, so sound buffering should be at least that long, and preferably synchronized to the framerate. On a DX and DXG, the 600x800 (not fullscreen) eink updates take about 650 msec to compete.

Okay I will have a head scratch about the maximum buffer size.

I think 4096 periods is the max size but I will have for figure out how long that actually is in the real world today.

In other news I only have a 3's and a DX so looks like I'll have to start saving up the pennies if I want to support all the devices. ho hum

I think - like you said - if I actually weld them together in one app and simply manage the error-handling in a very robust way - we will get better performance.

As for the Hint missing. That 99% likely to be the sym link to libsoundalibasound is either missing - or libsoundalibasound is not present at all. A sym link or x-compiled lib would fix it.

I'm going to keep playing about and see what I can get going today.

The actual code out of the first failed full-duplex demo above is pretty nice style so I think I will have an iterate over that and see if I can't back-port some of his error handling. - or possibly find an alternate improved - highly robust - way of splattering stuff the the card.

Thanks for playing. It is highly appreciated GM

[geekmaster]

Originally Posted by twobob

...
As for the Hint missing. That 99% likely to be the sym link to libsounda is either missing - or libsounda is not present at all. A sym link or x-compiled lib would fix it.
...
Thanks for playing. It is highly appreciated GM

You are welcome! Thank you for getting it working on the K3 (when CPU not overloaded).

"find /|grep libsounda" finds nothing on the K5, but also finds nothing on the K3?

Hmm...

Ahh... It finds libasound though...

On the K3:

[root@kindle root]# for i in $(find /|grep libasound);do ls -al $i;done 
lrwxrwxrwx    1 root     root           18 Sep 16  2010 /usr/lib/libasound.so.2 -> libasound.so.2.0.0
-rwxr-xr-x    1 root     root       755444 Jan 20  2008 /usr/lib/libasound.so.2.0.0
-rwxr-xr-x    1 root     root         9536 Nov  3  2011 /usr/lib/alsa-lib/libasound_module_pcm_retune.so.0.0.0
lrwxrwxrwx    1 root     root           36 Sep 16  2010 /usr/lib/alsa-lib/libasound_module_pcm_retune.so.0 -> libasound_module_pcm_retune.so.0.0.0
lrwxrwxrwx    1 root     root           36 Sep 16  2010 /usr/lib/alsa-lib/libasound_module_pcm_retune.so -> libasound_module_pcm_retune.so.0.0.0
-rwxr-xr-x    1 root     root       807384 Dec 17  2011 /mnt/us/mplayer/lib/libasound.so.2
-rwxr-xr-x    1 root     root       807384 Dec 17  2011 /mnt/base-us/mplayer/lib/libasound.so.2
[root@kindle root]#

On the K5:

[root@kindle us]# for i in $(find /|grep libasound);do ls -al $i;done
-rwxr-xr-x    1 root     root        482664 Nov  1 00:02 /usr/lib/libasound.so.2.0.0
lrwxrwxrwx    1 root     root            18 Nov  3 20:03 /usr/lib/libasound.so -> libasound.so.2.0.0
lrwxrwxrwx    1 root     root            18 Nov  3 20:03 /usr/lib/libasound.so.2 -> libasound.so.2.0.0
[root@kindle us]#

Interesting that the K3 (which works) is missing "/usr/lib/libasound.so.2", which exists on the K5 (which does not work)... Also, the K3 lib is 80K, but the K5 lib is only 48K -- perhaps it was built with less options enabled?

And I am curious why mplayer has its own libasound, when the K3 already has one...

[geekmaster]

Originally Posted by twobob

...
I think - like you said - if I actually weld them together in one app and simply manage the error-handling in a very robust way - we will get better performance.

I tend to GLUE code pieces together (with "glue code"). When you WELD them together, how do you keep the bits from catching fire?

[twobob]

Hmm. thinking again I'm not sure then

If I had one I would play. When I have a better grasp of how to sidestep more of the alsa funcs I will have more to offer I expect.

But I am now poorer after todays purchase of
The Audio Programming Book [Hardcover]
Richard Boulanger (Author), Victor Lazzarini (Author), Max Mathews (Author)
http://www.amazon.co.uk/Audio-Progra.../dp/0262014467 (oh happy day), so that will be nice to play with

Perhaps I will manage to find an old K5 / K4 one day

[knc1]

Originally Posted by geekmaster

I tend to GLUE code pieces together (with "glue code"). When you WELD them together, how do you keep the bits from catching fire?

Software Bug Spray is known for its ability to put out such fires.

[twobob]

Originally Posted by knc1

Software Bug Spray is known for its ability to put out such fires.

Okay thats odd.

With these settings:
usage: ./tones -f 4000 -D hw -c 2 -v -r16000 -mdirect_write

It runs at what seems like 1/4 speed and the cpu usage dives to like 6%

There might be something in this...

[twobob]

Yep.

I'm running the Geekmaster viewer in realtime.

and listening to the demo!! albeit at the weird speed. time to see if I can get this bugginess to work for us.

[geekmaster]

Originally Posted by twobob

Okay thats odd.

With these settings:
usage: ./tones -f 4000 -D hw -c 2 -v -r16000 -mdirect_write

It runs at what seems like 1/4 speed and the cpu usage dives to like 6%

There might be something in this

...

Yes. We can ENCODE the sound at quarter speed (at least for the gmvid audio). Hopefully that low overhead coupled with the K3 eink update high overhead will be less than 100%.

[twobob]

Originally Posted by geekmaster

Yes. We can ENCODE the sound at quarter speed (at least for the gmvid audio). Hopefully that low overhead coupled with the K3 eink update high overhead will be less than 100%.

You read my mind - hope you don't mind I x-posted before I read this.
But it seemed relevant to the other thread - and may energise interest I rationaled.

I'm actually quite excited about this @___@

[geekmaster]

I found a script on my k5 that decodes simultaneous audio and video:

#!/bin/sh
./ffmpeg -d -i $1 -an -pix_fmt gray -sws_flags neighbor -f rawvideo -s 800x600 -|./raw2gmv|./gmplay &
./ffmpeg -d -i $1 -vn -f s16le -acodec pcm_s16le -ar 44100 -|aplay -f cd

I do not think the K3 is powerful enough to do this realtime, with all that CPU time burned in the eink update syscalls.

Notice the "&" after the first ffmpeg call. They run as parallel processes.

BTW, there are lots of examples on the net using this "two instance" method for ffmpeg to play simultaneous audio and video streams. Luckily, interleaved input will already be cached for the other instance.

Also, notice that to make realtime playing possible, I cheated a little (nearest-neighbor pixel downsampling of HD rotated to 600x800).

[twobob]

Originally Posted by geekmaster

Notice the "&" after the first ffmpeg call. They run as parallel processes.

Yes, backgrounding is what I use [root@kindle video]# zcat gmvid.gmv.gz | ./gmplay & ../sound/tones -f 4000 -D hw -c 2 -v -r16000 -mdirect_write

(in that order as the video is finite and the tones are not)

I'm looking at my options in ffmpeg now

[geekmaster]

Using "neighbor" downsampling lets me play 1080p in real-time on a K5. The poor little K3 cannot even decode x.264 (or do significant downsampling for that matter) in real-time. I wanted minimal overhead in the play function so it could work on slow processors.

[twobob]

Originally Posted by geekmaster

Using "neighbor" downsampling lets me play 1080p in real-time on a K5. The poor little K3 cannot even decode x.264 (or do significant downsampling for that matter) in real-time. I wanted minimal overhead in the play function so it could work on slow processors.

oh okay. well I'll just trundle on and see what I get.

Pre-encoding files - then transfer / streaming them to the kindle doesn't seem to me that laborious...

And there is nothing preventing the setup of a real time host-side media encoder service. (Hardly rocket science - intercept request, setup ffmpeg stream (or w/e) - indicate to client stream ready. I have some one liner tcp code lying around if it comes to that.)

For me, this is about proving the K3 can play correctly encoded videos with sound in realtime. And look/sound good doing it.

Sure right now this is in it's infancy but why admit defeat just because the machine is at it's very limit. that's when it's most fun! So I shall rumble on


Thanks

[twobob]

To be clear I am not trying to encode the files on the kindle.

I'm trying to construct meaningful files that I can work with - perhaps with a modified raw2gmv - to create a stream for the gmPlayer++ (that can play a meaningful, striped audio stream alongside the video)

I hope that's nice and clear.