Audio (video) - Kindle. Getting it up and running, Code level.

[geekmaster]

Originally Posted by twobob

To be clear I am not trying to encode the files on the kindle.

I'm trying to construct meaningful files that I can work with - perhaps with a modified raw2gmv - to create a stream for the gmPlayer++ (that can play a meaningful, striped audio stream alongside the video)

I hope that's nice and clear.

Okay. It was always clear. I just wanted to provide information you may want to take into account.

I "standardized" on 600x800 7 FPS for eink video. Whatever size audio buffer you settle on should be interleaved every so many frames. The audio should probably precede the video a bit. To play on a DX, there should probably be about a second of audio buffered before the video it belongs to. The buffer should be large enough to avoid dropouts when the video falls behind. Up to a second video lag is allowed before framedropping starts in gmplay, so we need at least one second (plus a couple frames) of audio buffered up. IMHO.

[twobob]

Ah, thanks very much.

I thought I was not being clear.

So in essence either
a) a hand made file: Slice up two seconds of audio - put as header (and any other data you want dumping in there), buffer it, splice in second of video (say), then swap back to audio etc.

or

b) just get a known format and handle it in that way - i.e higher priority to audio consumption.

I like the sound of b) better.

One thought: I will try cramming some dummy "audio" through that demo via loading a wave first and swapping it into the buffer, see what results.

Start there come to think of it. (I asked the thread to be renamed as you suggested, thanks)

[twobob]

Originally Posted by NiLuJe

@twobob: The input twin of dmix is dsnoop, FWIW. Probably not that useful on the Kindle, but, hey . (You can glue a dmix output + dsnoop input together in a single pcm via an asym plug (See my desktop asoundrc as an example)).

Interesting you should say that.

running a strace on aplay does indeed seem to do some dsnoop jiggery-pokery.
link here: https://www.mobileread.com/forums/sho...9&postcount=42

I posted the strace on here above. I'll have a shufty around and see if dsnoop is an option when I have alsa playing fully nicely again.

Thanks

[geekmaster]

Originally Posted by twobob

Ah, thanks very much.

I thought I was not being clear.

So in essence either
a) a hand made file: Slice up two seconds of audio - put as header (and any other data you want dumping in there), buffer it, splice in second of video (say), then swap back to audio etc.

or

b) just get a known format and handle it in that way - i.e higher priority to audio consumption.

I like the sound of b) better.

One thought: I will try cramming some dummy "audio" through that demo via loading a wave first and swapping it into the buffer, see what results.

Start there come to think of it. (I asked the thread to be renamed as you suggested, thanks)

I agree that option b is "better" for a TOOL. My "codec" (dither and bitpacking) was designed to be rendered by extremely small and simple C code useful for a monolithic tutorial. Packing 8-bits/byte was really just a tiny evolution from packing two 4-bit pixels already used in the animation demos.

My video transcoder and player are just a tiny evolutionary step beyond my existing animation demos, designed to be easy to understand and reuse. Rather than designing a new container, a "real" player should just squeeze them into an output "driver" of a standard player (like ffmpeg or mplayer) that can play standard media containers (like MKV). But then of course, my super-simple codec just happens to compress extremely well with with gzip because the 8x8 dithering packs to 8-bit (LZ-friendly) boundaries. A "real" general purpose code may not compress kindle video quite so well.

[twobob]

Originally Posted by geekmaster

I agree that option b is "better" for a TOOL. My "codec" (dither and bitpacking) was designed to be rendered by extremely small and simple C code useful for a monolithic tutorial. Packing 8-bits/byte was really just a tiny evolution from packing two 4-bit pixels already used in the animation demos.

My video transcoder and player are just a tiny evolutionary step beyond my existing animation demos, designed to be easy to understand and reuse. Rather than designing a new container, a "real" player should just squeeze them into an output "driver" of a standard player (like ffmpeg or mplayer) that can play standard media containers (like MKV). But then of course, my super-simple codec just happens to compress extremely with with gzip because the dithering aligns to 8-bit (character) boundaries.

So in a nutshell, You think providing a stand-alone application is a good idea.

One that is the evolution of other stand-alone demos, which individually provide the capability to parse either standard media at really low rates and/or specially encoded media at device specific rates.

I'm not afraid getting my hands dirty on some "faux gui" work in the medium term. after all there is a video driver for petes sake, putting stuff on the screen is not a problem.

I am all for this. I personally would like to see more esoteric uses of the medium - I'm thinking real-time fft (on a 3!) and resulting graphics but that is another story and my headache. Thanks for the input, I'm feeling more confident this is actually attainable and possibly useful, and certainly impressive.

Time to get the welding gear out for some wave action

[geekmaster]

Originally Posted by twobob

So in a nutshell, You think providing a stand-alone application is a good idea.

One that is the evolution of other stand-alone demos, which individually provide the capability to parse either standard media at really low rates and/or specially encoded media at device specific rates.

I'm not afraid getting my hands dirty on some "faux gui" work in the medium term. after all there is a video driver for petes sake, putting stuff on the screen is not a problem.

I am all for this. I personally would like to see more esoteric uses of the medium - I'm thinking real-time fft (on a 3!) and resulting graphics but that is another story and my headache. Thanks for the input, I'm feeling more confident this is actually attainable and possibly useful, and certainly impressive.

Time to get the welding gear out for some wave action

I already converted some fft code for my demos. I do not remember what distracted me from it. I should go find it and pick up where I left off. I also have some raytracing code that worked well on eink (but slow). It was waiting for me to move all the chrome balls into a "geekmaster signature". And I have working mandelbrot (kindlebrot) code too. They all need some cleanup before publishing (hiding somewhere on my hard drive). I spend too much time reading and posting here and not enough coding. I wanted to pack them all together with a touchscreen/5-way joypad GUI to create a great demo. Someday...

[twobob]

Originally Posted by geekmaster

I already converted some fft code for my demos. I do not remember what distracted me from it. I should go find it and pick up where I left off. I also have some raytracing code that worked well on eink (but slow). It was waiting for me to move all the chrome balls into a "geekmaster signature". And I have working mandelbrot (kindlebrot) code too. They all need some cleanup before publishing (hiding somewhere on my hard drive). I spend too much time reading and posting here and not enough coding. I wanted to pack them all together with a touchscreen/5-way joypad GUI to create a great demo. Someday...

Well if you find your way clear to having a look at cramming some audio through the buffer created by that demo running with the params I indicated, I am certain you could achieve in 20 mins that which will take me a week.

I see myself as the poor guy giving at the temple, I have almost nothing to give but I'm happy to give it all.

I bought the full version of that book in the hopes that there are a few example gems in there for optimised audio crunchery. Read my way through much of the crunchery links you provided. People will be impressed to see a+v running on the 3 (I have a dxg so I can test on that too)

I know all the people who saw your early demos around here were mind-blown and giving them access to something they understand - like videos with sound - is the first step to acceptance and uptake.

I see your drivers as a genuine way to get all manner of info to the screen and coupled with - audio full duplex, keyboard and touch / gesture support / motion support - it's a goshdamned staggeringly powerful proposal, used correctly.

My hats off to you,

[geekmaster]

Okay, I ran this on my K3 with no apparent video slowdowns:

zcat clock.gmv.gz|gmvid/gmplay & ./tones -f4000 -Dhw -c2 -v -r16000 -mdirect_write

Sound and video playing nicely together (with no black eyes, bloody lips, or skinned knuckles).

[twobob]

Originally Posted by geekmaster

Okay, I ran this on my K3 with no apparent video slowdowns:

zcat clock.gmv.gz|gmvid/gmplay & ./tones -f4000 -Dhw -c2 -v -r16000 -mdirect_write

Sound and video playing nicely together (with no black eyes, bloody lips, or skinned knuckles).

Happy days! I am reconfigging my ffmpeg from source.

apparently it wants to be --disable-decoder=amrnb or it refuses to encode stuff. sigh.
I'm throwing the kitchen sink in while I am at it, might as well I suppose.

All day to get a file I can parse. sigh. lol.

[geekmaster]

Now that there is audio, I suppose I should resurrect some of my early algorithmic audio experiments that were done on a SC/MP microprocessor in about 1974 (before sound cards -- only 1-bit output). People at work borrowed all my audio tapes and never gave them back.

People said they were hypnotic and mesmerizing, and they wanted to borrow them so friends and family could hear them.

[geekmaster]

Originally Posted by twobob

Happy days! I am reconfigging my ffmpeg from source.

apparently it wants to be --disable-decoder=amrnb or it refuses to encode stuff. sigh.
I'm throwing the kitchen sink in while I am at it, might as well I suppose.

All day to get a file I can parse. sigh. lol.

My ffmpeg that runs on all my kindles (static compiled) includes all build options EXCEPT alsa sound. So why not throw in the kitchen sink in the audio department too.

If you could make a static linked version so it will run on my DX and K5, that would be sweet!

I detest library dependencies.

[twobob]

Originally Posted by geekmaster

Now that there is audio, I suppose I should resurrect some of my early algorithmic audio experiments that were done on a SC/MP microprocessor in about 1973-1974 (before sound cards -- only 1-bit output). People at work borrowed all my audio tapes and never gave them back.

People said they were hypnotic and mesmerizing, and they wanted to borrow them so friends and family could here there.

Lol. I went out an bought an album of mine on 12" the other day from a collectors shop.

I had "lost" all of the original copies. fortunately mine was available to purchase back!
The countless hours of lost work in ones life doesn't bear thinking about

That said, some of the largest things come from the most unlikely of sources so it pays to spread yourself thin I would love to hear some procedural stuff. My last project was designing effects processors in csound, cross compiled to run as standalone bins via ladspa. pretty cool stuff all in and comparable processes in creation once the guts are up and running.

I'm getting there. inch by inch. more demos would be awesome. definately Toppost them

[twobob]

So a bit more testing. What does the overhead of running the audio mean in terms of performance visually? Well, this is the K3 we are talking about so it's not like watching an IMAX but the performance while running the sound comes out at a perfectly acceptable:

861 frames in 149.8 secs = 5.7 FPS

Whereas without the sound it weighs in around:

969 frames in 149.8 secs = 6.5 FPS

Showing us that a few frames were dropped but meh, it was not perceivable to my eyes thanks to GM's very tidy drop frame code. This gives me enough hope to try and squash some waves through the demo. and possibly - glue - the video code into my audio loop.

[twobob]

this is an interesting project:

http://hhallgrimur.wordpress.com/code/ And https://github.com/jxxcarlson/sf2sound

================================================== ===============
SF2A README: 4/13/2011
================================================== ===============

DESCRIPTION
-----------
sf2a is a unix command-line program for creating sound files from text such as

moderato: do re mi.

might have another look at that at some point. noted here for posterity

Also cool is:

http://hhallgrimur.wordpress.com/cod...nce=b2b25cb8c2

take file of quadruples (freq, dur, decay, amplitude) as input, produce file of sampled waveform as output Wav.

[twobob]

Originally Posted by geekmaster

That looks like a fun little project. I often find that interleaving the code into one loop uses less CPU that trying to multiprocess it as two apps, or other coprocessing methods that rely on two simultaneous loops....

...Anyway, I am looking forward to adding some some code to gmplay, to play gmv video files that contain interleaved audio and video (using your code above). Thanks. Just need to get caught up at work a bit first before I take some "educational play time".

Not sure how I missed this post.

Understood, In the interim I shall be assassinating the project in my own inimitable style. Good luck on the catching up.