View Single Post
Old 05-04-2012, 04:04 PM   #1
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Arrow geekmaster kindle video player

UPDATE: We now have the ability to play streaming video on the kindles directly from the internet, with the help of a host PC running ffmpeg. The goal now is to port ffmpeg to the kindle, and add a nice GUI wrapper on the kindle to make this user-friendly. For now, the easy way is to play pre-encoded videos as shown below. The video transcoder used to convert raw video to .gmv format can be found here: https://www.mobileread.com/forums/sho...79#post2074379


geekmaster's kindle video player accepts raw video piped into STDIN. To play sample video:

zcat gmvid.gmv.gz|./gmplay
..........^--(replace RED text from list below)


Sample video download links are provided below. These videos can be played on all eink kindles with this gmplay program. DX and DXG kindles drop frames, but the sequence of still images is clearer and sharper than on other kindle models.

Geek techno:
Spoiler:
Caveats: Video suitable for eink must be designed (or filtered) to be compatible with long-persistence displays. This means that we need to minimize the number of pixels that have high constrast changes between frames, to minimize ghosting and smearing visual artifacts. Also, it helps to have a slowly changing background like this demo shows, to help erase ghosting. Without the moving background, that bouncing ball in the sample video would leave permanent light shadows on dark background areas. I adjusted the video for best eink presentation, and I plan to do the same for future kindle video game designs. For "live" video, this should work well for some video content (such as talk shows) and not so well for others (such as action movies). Fast camera panning is bad for long-persistence video display devices like eink. So choose your eink video viewing habits wisely.

Tech details: The "gmvid.gmv.gz" sample video contains computer generated animation created with a program that I ran on my linux PC, which outputs raw 600x800 framebuffer data AFTER dithering to pure black and white, then packed lsb-first to 8-pixels per byte and compressed with gzip. There are more efficient video codecs, but this is fast, lossless (needed for moiré effects used in this video), simple (only a few lines of new code), and works well on eink kindles.

News: I also have a video filter I wrote that goes between mencoder (linux video encoder) and gzip, that creates "gmplay-compatible" video, but I am having issues with mencoder inserting AVI container crap into its "raw" output. My internet streaming video looks good on the kindles at its raw captured 320x240 resolution, but has problems with frame synchronization and drift (probably from inserted AVI crap), so it is not ready for release yet...

Things to do later:I noticed jerky animation when I ran my "animation generator" program on my K3 piped directly into gmplay on my K3, and I think that moving the eink delay (while loop) from inside gmlib() to just after the fread() call so reading from a slow video source occurs while "unsafe to write to fb0" instead of while "safe to write to fb0", giving more time for video decoding on a busy or underpowered kindle. The K4 and K5 are faster and do not have that problem. My video generator that created the sample video file outputs to STDOUT instead of /dev/fb0, so it runs on my host linux PC and the kindles with NO CHANGES to the source code -- how cool is that? My dithering is much faster now that I converted my dither tables to logical expressions using Karnaugh maps (cached code is MUCH faster than table lookups in RAM). I cannot find anything like my method on Google -- perhaps instead of publishing it I should patent it instead?

News: New version 1.2 supports all eink kindles, tested on DX,DXG,K3,K4main,K4diags,K5main,K5diags. Frame dropping added so slower (1.5 FPS) DX and DXG play video at the same speed as newer kindles. The large-screen DX and DXG display the 600x800 video centered in a black border.

Hmm... I changed where the delay is to overlap the wait for eink update with the file read, expecting that to make the K3 run faster with it was playing video piped in from a program that outputs video (both running on the same K3 at the same time).

Playing the sample video from a takes 76.5 seconds on all kindles that can run fast enough (including the K3). Before the speed optimization (overlapped wait and read), it ran full speed on K4 and K5, but on K3 it tool 90.9 seconds.

Now after optimizing it, doing the same test takes 175.6 seconds. Huh? That is only half the speed of the previous worst case. This is why you need to use instrumented timing tests on modern computers. Things that logically SHOULD run faster sometimes run MUCH slower. The currently recommended way to optimize a code function is to write it and time it all ways, and use the one that is fastest. Human logic and intuition fail miserably when trying to speed-optimize multi-level cache systems. Oh well...

Progress report: At least the K4 main boot is not displaying a negative image anymore, and this now works on the DX and DXG (at 600x800 resolution). Unfortunately, the eink updates only return from the system calls after wating about 600 msec, so you get only 1.5 FPS on the DX and DXG. After implementing "wait for vertical sync" delays to slow down fast kindles to run at K3 speed, it looks like I will now have to add frame dropping for older kindles, so they can finish a video in the same amount of time as a K3. Moving objects will move at the same speed, but they will move farther between eink updates. At least that is the plan. For now, I will update the code in the first post to the latest mplay-1.2.

I added frame dropping if kindle is falling behind, so the video now finishes in the same amount of time on all kindles. New kindles are slowed down by waiting for next VSYNC time, and old kindles are sped up by dropping frames when they fall behind. I plan to add command line options to disable frame drop and adjust the maximum allowed frames per second. I should probably add a 600x800 header frame to the video to contain things like recorded display resolution and framerate, and perhaps bits per pixel and more. Keeping it the same size as a current frame will make it work on older versions of the player (and the first "header frame" could contain a copy of the first frame of video with only a few pixels changed to contain "data"). At least, that is my plan.

V1.3 notes: New version 1.3 supports DX and DXG, fixes negative video on K4main, and fixes an "8-bit half-width" bug added to 1.2 when adding a "last minute" optimization for tcc.

V1.4 notes: On v1.3 the K3 is sometimes dropping frames (not unexpected -- it DOES get a little behind then catches up -- eink updates take variable amounts of time when we are updating 2.5x what the book says it can do (i.e. eink "overclocking" ). I should probably only drop frames when multiple frames behind... I am overclocking by using asynchronouse (non-blocking) calls and ignoring return codes, then delaying for best speed and image quality, which gives 2.5x on K3 or up to 15x on K5 (with bad smearing on fast-moving objects). *** New v1.4 only drops frames when more than 1 second behind. The K3 dropped 4 frames instead of 100 frames like before, and I did not even notice the missing frames. Cool. Now to test on DX again before posting the update. (Then sleep?). Wow -- K4main is dropping between 102 and 104 frames on each run, and yet the motion is smooth and I cannot see where it is dropping them -- excellent code tuning job!

V1.4a update and summary: New version 1.4a adds FBSIZE constant for tcc compiler. Updates since v1.3a include much better frame dropping support, for when the kindle "gets busy" (wifi phone-home, screensaver mode change, document indexing, etc.). The K4main "hybrid" eink mode routinely drops about 100 frames during the sample video, but I cannot see any jerky motion showing where they are getting dropped, which indicates a sweet spot in vsync and framedropping, so we are done with the code tuning! The K4 normally drops 4 frames (probably at the frame after the first fade to black), but when compiled with tcc it the K3 drops about 100 frames (but also not noticed with the new v1.4 frame dropping updates).

Because we are updating the eink display 2.3 times faster than the 300ms specified in the Reference Manual, the eink drivers can (rarely) fall so far behind they cannot catch up (during high background activity). In the event of a rare worst-case event the display quality may suffer until the end of the video -- in that case stop the video and restart it for good quality smooth animation. 2.3x eink overclocking is cool. How else can we do the "impossible"?

Version 1.5 update: New version 1.5 fixes K5 firmware 5.1.0 ioctl() calls, which did not work because 5.1.0 changed a critical internal structure than breaks compatibility with previous kindles and makes the mxcfb.h header file incompatible between different kindle models. The eink header files are no longer used here.

Version 1.5a: "arm-linux-gcc -Wall -Wextra -pedantic -O3 -o gmplay gmplay-1.5a.c" only warns about using C++ style comments (and removing -pedantic fixes that "problem".)

More things I want to add: Although I have not taken the time to implement them yet (baby steps, you know, or I get confused ), I know how to extend my dither logical expressions to work on 32-bits (4 pixels) in parallel (SIMD Within A Register) using the same logic I am using now. They key is that if I sacrifice a bit (127 shades of gray) I can shift 4 pixels in a 32-bit word right 1, mask off the sign bits (in all 4 bytes), and after adding the 4 combined dither threshold bytes, all 4 sign bits will become dithered pixels:
((px32/2&0x7f7f7f7f)+dt32&0x80808080)/128*255
and you should have the 4 sign bits each extended to 8 0-bits or 8 1-bits). Of course, that is all in my head and needs testing. I have not actually done this before, you know...

Oh... and I figured out how to get fluid animation (no longer limited to 1.5 FPS) on a DX and DXG now too...

Magic revealed:
Spoiler:
gmplay - geekmaster's kindle video player
PHP Code:
//====================================================
// gmplay 1.5a - geekmaster's kindle video player
// Copyright (C) 2012 by geekmaster, with MIT license:
// http://www.opensource.org/licenses/mit-license.php
//----------------------------------------------------
// Tested on DX,DXG,K3,K4main,K4diags,K5main,K5diags.
//----------------------------------------------------
#include <sys/ioctl.h>     // ioctl
#include <sys/mman.h>     // mmap, munmap
#include <stdio.h>       // printf
#include <stdlib.h>     // malloc, free
#include <linux/fb.h>  // screeninfo
#include <sys/time.h> // gettimeofday
#include <unistd.h>  // usleep
#include <string.h> // memset, memcpy
#include <fcntl.h> // open, close, write
#include <time.h> // time
typedef unsigned long u64;
typedef unsigned int u32;
typedef unsigned short u16;
typedef unsigned char u8;
u32 __invalid_size_argument_for_IOC// ioctl.h bug fix for tcc
//----- eink definitions from eink_fb.h and mxcfb.h -----
#define EU3 0x46dd
#define EU50 0x4040462e
#define EU51 0x4048462e
struct update_area_t {int x1,y1,x2,y2,which_fx;u8 *buffer;};
struct mxcfb_rect {u32 top,left,width,height;};
struct mxcfb_alt_buffer_data {u32 phys_addr,width,height;
    
struct mxcfb_rect alt_update_region;};
struct mxcfb_update_data {struct mxcfb_rect update_region;
    
u32 waveform_mode,update_mode,update_marker;int temp;uint flags;
    
struct mxcfb_alt_buffer_data alt_buffer_data;};
struct mxcfb_update_data51 {struct mxcfb_rect update_region;
   
u32 waveform_mode,update_mode,update_marker;
   
u32 hist_bw_waveform_mode,hist_gray_waveform_mode;
   
int temp;uint flags;struct mxcfb_alt_buffer_data alt_buffer_data;};
//----- function prototypes -----
void gmplay4(void);
void gmplay8(void);
int getmsec(void);
int gmlib(int);
//----- gmlib global vars -----
enum GMLIB_op {GMLIB_INIT,GMLIB_CLOSE,GMLIB_UPDATE,GMLIB_VSYNC};
u8 *fb0=NULL;     // framebuffer pointer
int fdFB=0;      // fb0 file descriptor
int teu=0;      // eink update time
u32 fs=0;      // fb0 stride
u32 MX=0;     // xres (visible)
u32 MY=0;    // yres (visible)
u32 VY=0;   // (VY>MY): mxcfb driver
u8 ppb=0;  // pixels per byte
u32 fc=0// frame counter
#define FBSIZE (600/8*800)
//==================================
// gmplay4 - play video on 4-bit fb0
//----------------------------------
void gmplay4(void) {
    
u32 i,x,y,b,p,off=(MY/2-400)*fs+MX/4-150,fbsize=FBSIZEu8 fbt[FBSIZE];
    while (
fread(fbt,fbsize,1,stdin)) { teu+=130// teu: next update time
        
if (getmsec()>teu+1000) continue; // drop frame if > 1 sec behind
        
gmlib(GMLIB_VSYNC); // wait for fb0 ready
        
for (y=0;y<800;y++) for (x=0;x<600;x+=8) {
            
b=fbt[600/8*y+x/8]; i=y*fs+x/2+off;
            
p=(b&1)*240b>>=1fb0[i]=p|(b&1)*15b>>=1;
            
p=(b&1)*240b>>=1fb0[i+1]=p|(b&1)*15b>>=1;
            
p=(b&1)*240b>>=1fb0[i+2]=p|(b&1)*15b>>=1;
            
p=(b&1)*240b>>=1fb0[i+3]=p|(b&1)*15;
        } 
fc++; gmlib(GMLIB_UPDATE);
    }
}
//==================================
// gmplay8 - play video on 8-bit fb0
//----------------------------------
void gmplay8(void) {
    
u32 i,x,y,b,fbsize=FBSIZEu8 fbt[FBSIZE];
    while (
fread(fbt,fbsize,1,stdin)) { teu+=130// teu: next update time
        
if (getmsec()>teu+1000) continue; // drop frame if > 1 sec behind
        
gmlib(GMLIB_VSYNC); // wait for fb0 ready
        
for (y=0;y<800;y++) for (x=0;x<600;x+=8) {
            
b=fbt[600/8*y+x/8]; i=y*fs+x;
            
fb0[i]=(b&1)*255b>>=1fb0[i+1]=(b&1)*255b>>=1;
            
fb0[i+2]=(b&1)*255b>>=1fb0[i+3]=(b&1)*255b>>=1;
            
fb0[i+4]=(b&1)*255b>>=1fb0[i+5]=(b&1)*255b>>=1;
            
fb0[i+6]=(b&1)*255b>>=1fb0[i+7]=(b&1)*255;
        } 
fc++; gmlib(GMLIB_UPDATE);
    }
}
//====================================
// gmlib - geekmaster function library
// op (init, update, vsync, close)
//------------------------------------
int gmlib(int op) {
    static 
struct update_area_t ua={0,0,600,800,21,NULL};
    static 
struct mxcfb_update_data ur={
        {
0,0,600,800},257,0,1,0x1001,0,{0,0,0,{0,0,0,0}}};
    static 
struct mxcfb_update_data51 ur51={
        {
0,0,600,800},257,0,1,0,0,0x1001,0,{0,0,0,{0,0,0,0}}};
    static 
int eupcode; static void *eupdata=NULL;
    
struct fb_var_screeninfo screeninfo;
    if (
GMLIB_INIT==op) { teu=getmsec(); fdFB=open("/dev/fb0",O_RDWR);
        
ioctl(fdFB,FBIOGET_VSCREENINFO,&screeninfo);
        
ppb=8/screeninfo.bits_per_pixelfs=screeninfo.xres_virtual/ppb;
        
VY=screeninfo.yres_virtualMX=screeninfo.xresMY=screeninfo.yres;
        
ua.x2=MXua.y2=MYur.update_region.width=MXur.update_region.height=MY;
        
fb0=(u8 *)mmap(0,MY*fs,PROT_READ|PROT_WRITE,MAP_SHARED,fdFB,0); // map fb0
        
if (VY>MY) { eupcode=EU50eupdata=&urur.update_mode=0;
            if (
ioctl(fdFB,eupcode,eupdata)<0) { eupcode=EU51eupdata=&ur51; }
        } else { 
eupcode=EU3eupdata=&ua; }
        
system("eips -f -c;eips -c"); sleep(1);
    } else if (
GMLIB_UPDATE==op) {
        if (
ioctl(fdFB,eupcode,eupdata)<0system("eips ''");  // 5.1.0 fallback
    
} else if (GMLIB_VSYNC==op) { while (teu>getmsec()) usleep(1000); // fb0 busy
    
} else if (GMLIB_CLOSE==op) { gmlib(GMLIB_UPDATE); sleep(1); // last screen
        
system("eips -f -c;eips -c"); munmap(fb0,MY*fs); close(fdFB);
    } else { return -
1; }
    return 
0;
}
//====================================
// getmsec - get msec since first call
// (tick counter wraps every 12 days)
//------------------------------------
int getmsec(void) {
    
int tc; static int ts=0struct timeval tv;
    
gettimeofday(&tv,NULL); tc=tv.tv_usec/1000+1000*(0xFFFFF&tv.tv_sec);
    if (
0==tsts=tc;
    return 
tc-ts;
}
//==================
// main - start here
//------------------
int main(void) {
    
int i;
    
gmlib(GMLIB_INIT);
    if (
ppb-1) { gmplay4(); } else { gmplay8(); }
      
i=getmsec()/100printf("%d frames in %0.1f secs = %2.1f FPS\n",
        
fc,(double)i/10.0,(double)fc*10.0/i);
    
gmlib(GMLIB_CLOSE);
    return 
0;

Notes: Version 1.5 restores ioctl() update calls that broke in 5.1.0 firmware, by copying the eink structures from the 5.0.x and 5.1.x gpl code and renaming the new one to fix the name conflict with the old one. Now that they made the 5.1.x mxcfb.h header file incompatible with the 5.0.x version, I decided to stop using their broken eink header files.


Geekmaster Signature Video Productions:

Geekmaster Video #1 (12MB, gmvid.gmv.gz): [attached file below]
Geekmaster's signature computer animation, featuring moiré effects, bouncing ball, and more.

K3 Unbrick (13MB, k3unbrick.gmv.gz): http://www.mediafire.com/?q1m816pqcpcrcns
Seaniko7's great Kindle 3 unbricking (debricking) video tutorial.

Living FPV 2 (31MB, fpv2.gmv.gz): http://www.mediafire.com/?b0825b6v27ethw2
One of Trappy's "less infamous" First Person Video R/C adventure films.

Big Buck Bunny (55MB, bunny.gmv.gz): http://www.mediafire.com/?n2p05m3lufjz6kg
Open source animated short film featuring a big fluffy bunny and rodent revenge.

Sintel (60MB, sintel.gmv.gz): http://www.mediafire.com/?kb4w2k5sdwazlo9
Open source animated short film featuring a girl, a dragon, a quest (non-motion blur version).

Safety Last (62MB, clock.gmv.gz): http://www.mediafire.com/?3vwwybcvtkc1i02
Harold Lloyd & Buster Keaton iconic silent short comedy involving a clock and a tall building.

Great Train Robbery (70MB, gtr.gmv.gz): http://www.mediafire.com/?bsj358i7mud9i5u
Edison Films, 1903. The first film that told a story (and first "Western", very popular).

Sand Art (51MB, sandart.gmv.gz): http://www.mediafire.com/?n3ldlih1uuxqapx
Kseniya Simonova (Ukraine) performs live animation with sand and light.

Monkey Island (8MB, monkey.gmv.gz): http://www.mediafire.com/?eruly5mu909asrl
Trailer for "Tales of Monkey Island" video game.

Panspermia (19MB, panspermia.gmv.gz): http://www.mediafire.com/?6wukqdh662xrs32
Siggraph 90, Karl Sims Thinking Machines animation (with kindle motion blur).

Harvey (34MB, harvey.gmv.gz): http://www.mediafire.com/?8aqfq6yxkabyqt4
(Siggraph 2001) Creepy horror flick using advance CGI techniques. Mature audiences only!


If you play these videos on a Kindle 4, they play a lot better (less jerky) when you boot to diagnostics mode and run from SSH (which requires some techical knowledge that can be learned in this forum), but even running after booting normally this is a lot better than you could do on a K4 before now.
Attached Files
File Type: gz gmvid.gmv.gz (12.01 MB, 1632 views)
File Type: gz gmplay-1.5a.tar.gz (9.0 KB, 1798 views)
File Type: zip Videos-KUAL-EXTENSION.zip (12.02 MB, 1665 views)

Last edited by twobob; 09-12-2013 at 11:58 PM. Reason: Add more videos!
geekmaster is offline   Reply With Quote