geekmaster kindle video player

[geekmaster]

ffmpeg was a PITA to cross-compile correctly. I suppose I should post my binary, which has almost everything enabled. I had to leave ALSA out, but you can pipe the output to aplay for sound playback.

Anyway, the gmplay video has a 130msec average framerate, with a variable latency up to 1 second. Synchronizing sound to that will be difficult, and syncing video to sound will cause a lot more dropped frames. I needed to allow up to 1 second lag before framedropping to prevent too many dropped frames. The eink drivers speed up and slow down depending on scene complexity. 7.8 FPS is smooth enough on eink, but dropped frames when it cannot quite keep up cut that in HALF, which looks terrible. I chose to allow 1 second latency instead, which looks nice. But sound sync will be difficult.

A choice of nice sound with jerky video, or smooth video with glitchy shound, is a difficult decision. My thoughts are to implement variable audio playback speed with a 3-phase ring buffer with phase fadeout across boundary transitions (similar to changing TTS playback speed). That way we can vary sound playback speed while maintaining pitch accuracy, to sync sound to the laggy video. So the choices are complex "DSP" code, or simple code with either glitchy audio or glitchy video. I like KISS, but complexity wins in this case... Some day RSN.

[twobob]

or just wait for me to build one via buildroot...

[NiLuJe]

Also, don't use a hardfloat toolchain on the Kindle (any of them). You're bound to run into some issues, given that the Kindle uses a softfp TC, especially with something FP heavy like ffmpeg/libav.

[twobob]

oh.... well that was bloody timely information I can tell you.

so... err. toolchain options are limited therefore to Linaro <= 2012.04
(as later ones REQUIRE hardfloat to work on buildroot)

I do have one question about that? Isn't that like... really stupid? I thought ARM8,9 LOVED hardfloat... why would they use a softlfoat one?

sigh. (goes off to reconfig buildroot)


YES:

Linaro toolchain for the ARM architecture. It uses Linaro
GCC 2012.04 (based on gcc 4.7), Linaro GDB 2012.04 (based on
GDB 7.4), eglibc 2.13. It generates code that runs on all
Cortex-A profile devices, but tuned for the Cortex-A9. The
code generated is Thumb 2, with the softfp calling
convention, and uses the VFPv3-D16 FPU instructions.

And

No:

Linaro toolchain for the ARM architecture. It uses Linaro
GCC 2012.05 (based on gcc 4.7), Linaro GDB 2012.05 (based on
GDB 7.4), eglibc 2.13. It generates code that runs on all
Cortex-A profile devices, but tuned for the Cortex-A9. The
code generated is Thumb 2, with the hard floating point
calling convention, and uses the VFPv3-D16 FPU instructions.

To use this toolchain, you must disable soft float usage.

got you. thanks

[NiLuJe]

@twobob: Err, any idea *why* you can't use a softfp TC with BR with a newer Linaro release?

I get that it's the turning point release when they switched the loader name for the multilib/hardf stuff, but it's working pretty well with a no multilib softfp setup [nothing's changed there] :? (My latest batch of stuff was built with Linaro 2012.09).

[knc1]

Keep in mind that Buildroot is a tool for contractors that are building **new** firmware images.

You are doing **re-work** of an existing firmware image.

And until we "port" Tizen to the Kindles, we don't get to do our own **new** work.

[NiLuJe]

@twobob: Aren't they speaking about Linaro's *binary* releases?

(Also, FWIW, the K5 is based around eglibc 2.12, which tremendously helps in being able to use a decent, recent TC without running into ABI mismatches every tuesdays ).

[geekmaster]

Originally Posted by twobob

... why would they use a softlfoat one? ...

I read somewhere that the hardfp can actually be SLOWER than softfp. The only real reason to use hardfp is for parallel processing (letting the CPU do stuff while floats are being processed in hardware). That is probably dependent on particular arm implementations though...

[twobob]

Originally Posted by NiLuJe

@twobob: Err, any idea *why* you can't use a softfp TC with BR with a newer Linaro release?

I get that it's the turning point release when they switched the loader name for the multilib/hardf stuff, but it's working pretty well with a no multilib softfp setup [nothing's changed there] :? (My latest batch of stuff was built with Linaro 2012.09).

I did try to dig around in the "commits" for a "reason". all I found was those notes...

generally in BR (this is my experience - not fact) they simply add those notes until they work around whatever issue is preventing it working.

Spoiler:

I shall go hunt down the relevant BR-mailing-list thread when I ACTUALLY FIGURE OUT HOW TO BACKUP MY KINDLE

[knc1]

Originally Posted by NiLuJe

@twobob: Aren't they speaking about Linaro's *binary* releases?

YES.

But Buildroot can be configured to run crosstool-ng as its "tool chain generator" - and that can build a <whatever> configuration from Linaro (or other) sources.

A note to someone else here: Buildroot will run on MacOSx.
It currently requires a single, one-line, patch to the mktmp call, see the BR m.l. recent postings.

[knc1]

Originally Posted by geekmaster

I read somewhere that the hardfp can actually be SLOWER than softfp. The only real reason to use hardfp is for parallel processing (letting the CPU do stuff while floats are being processed in hardware). That is probably dependent on particular arm implementations though...

That information probably dates back to VFP (actually, VFP1.1 which is the first one that hit silicon).

The K4&K5 have the newer VFP3 co-processor.

[twobob]

hmm.. I'll go whip up a crosstool-ng RSN then see what that yields...
thanks all

and when I do - I'll check it with the EXCELLENT GM video code...

Just need to mangle that into one of the BR apps...

[twobob]

ffmpeg -i http://bit.ly/RhDLBu -vn -acodec pcm_s16le -f s16le - | nc -l 5556

helpful

[root@kindle lib]# nc 192.168.1.65 5556|aplay -f cd

[ppoo]

I run ffmpeg+raw2gmv+gmplay on Kindle but it show only blank screen.
Raw2gmv and gmplay work well ( I've tested they seperately).

[root@kindle bin]# ./ffmpeg -i http://archive.org/download/SevenWiseDwarfs/Seven
WiseDwarfs.mp4 -pix_fmt gray -vcodec rawvideo -f rawvideo -r 7.7 -y -s 800x600 -
| ./raw2gmv | ./gmplay15
ffmpeg version N-44781-g299c0b3 Copyright (c) 2000-2012 the FFmpeg developers
  built on Sep 26 2012 16:01:09 with gcc 4.7.2 (crosstool-NG 1.16.0) 20120701 (prerelease)
  configuration: --enable-cross-compile --cross-prefix=/root/sbox2/x-tools/arm-cortex_a8-linux-gnueabi/bin/ --sysroot=/root/sbox2/x-tools/arm-cortex_a8-linux-gnueabi/arm-cortex_a8-linux-gnueabi/sysroot --host-cc=/usr/bin/gcc --arch=arm --target-os=linux --extra-cflags=-fPIC --enable-static --enable-shared --prefix=/home/ppoo/Downloads/ffmpeg/kindle --disable-doc --enable-gpl --enable-nonfree --enable-ffmpeg --enable-ffplay --enable-ffserver --enable-postproc --enable-swscale --enable-indevs --enable-outdevs --enable-pthreads --disable-zlib --enable-armv6 --enable-neon --enable-armvfp
  libavutil      51. 73.101 / 51. 73.101
  libavcodec     54. 59.100 / 54. 59.100
  libavformat    54. 28.101 / 54. 28.101
  libavdevice    54.  2.101 / 54.  2.101
  libavfilter     3. 17.100 /  3. 17.100
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'http://archive.org/download/SevenWiseDwarfs/SevenWiseDwarfs.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomavc1mp42
    creation_time   : 1940-04-17 04:27:17
  Duration: 00:03:49.89, start: 0.000000, bitrate: 427 kb/s
    Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, s16, 134 kb/s
    Metadata:
      creation_time   : 1940-04-17 04:27:17
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
    Stream #0:1(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 290 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      creation_time   : 1940-04-17 04:27:17
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
Output #0, rawvideo, to 'pipe:':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomavc1mp42
    encoder         : Lavf54.28.101
    Stream #0:0(und): Video: rawvideo (Y800 / 0x30303859), gray, 800x600 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 7.70 tbc
    Metadata:
      creation_time   : 1940-04-17 04:27:17
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
Stream mapping:
  Stream #0:1 -> #0:0 (h264 -> rawvideo)
Press [q] to stop, [?] for help
frame=    4 fps=0.0 q=0.0 size=    1875kB time=00:00:00.51 bitrate=29568.1kbits/frame=    6 fps=5.9 q=0.0 size=    2812kB time=00:00:00.77 bitrate=29568.0kbits/frame=    8 fps=5.1 q=0.0 size=    3750kB time=00:00:01.03 bitrate=29568.1kbits/frame=   10 fps=4.8 q=0.0 size=    4688kB time=00:00:01.29 bitrate=29568.0kbits/frame=   12 fps=4.5 q=0.0 size=    5625kB time=00:00:01.55 bitrate=29568.0kbits/frame=   14 fps=4.3 q=0.0 size=    6562kB time=00:00:01.81 bitrate=29568.1kbits/frame=   16 fps=4.2 q=0.0 size=    7500kB time=00:00:02.07 bitrate=29568.0kbits/s dup=0 drop=36

[twobob]

seems to me that would be the dropping code in GM lib.

That is to say the FB gets filled too late to ever be written to the screen and the same thing happens over and over again in each pass.


hmm not the dropping code... well not according to it's own readouts... "drop=0"

I'll have more of a play, I've chosen a much simpler "source", I'll try to get that going:

Then ramp up the "real-time complexity of operation" required of ffmpeg incrementally by providing "tougher" (larger perhaps, whatever turns out to be "tougher") sources.

Sounds a likely stress test path. (not withstanding getting something on the screen)

EDIT:

Possibly to do with the piping for example... take ¦ raw2gmv out and you will see the stuff pipe through...

so maybe it's the latency of two pipes? possibly it would be better to embed raw2gmv code directly in gmplay and have a gmRawPlay inteads.

Then we could test better what effects the extra overheads incurred in transcoding have.

err.. more thinking required : )