Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle > Kindle Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 07-11-2012, 02:12 PM   #136
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by twobob View Post
... I'm just struggling to visualise exactly the right settings in ffmpeg.

Ah well I'm playing now, Ill figure it out
I too aim to evolve the demos to be more robust and informative and the audio thread to contain at least enough data to get most audio requiring apps off the ground.
Look at the script posted above that decodes audio and video with ffmpeg and pipes them to raw2gmv|gmplay and to aplay.

Real tools indeed need to be robust. But tutorial demos need to be crystallized into their bare essence with no distracting baggage (like error handling that can easily swamp the inner algorithm). I am also a big fan of getting as much of the important stuff "above the fold" (no scrolling) as possible, which means minimal white space. You can always expand it with AStyle later...
geekmaster is offline   Reply With Quote
Old 07-11-2012, 03:24 PM   #137
twobob
( ͡° ͜ʖ ͡°){ʇlnɐɟ ƃǝs}Týr
twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.
 
twobob's Avatar
 
Posts: 6,586
Karma: 6299991
Join Date: Jun 2012
Location: uti gratia usura (Yao ying da ying; Mo ying da yieng)
Device: PW-WIFI|K5-3G+WIFI| K4|K3-3G|DXG|K2| Rooted Nook Touch
Quote:
Originally Posted by geekmaster View Post
Look at the script posted above that decodes audio and video with ffmpeg and pipes them to raw2gmv|gmplay and to aplay.
well yes. but i want a unified format surely to parse.

Ill have look what my options are

EDIT: I'm looking.
And I noticed this: If your computer is not fast enough, you can speed up the compression at the expense of the compression ratio. You can use ’-me zero’ to speed up motion estimation, and ’-g 0’ to disable motion estimation completely (you have only I-frames, which means it is about as good as JPEG compression).

Which I thought I would mention.

Last edited by twobob; 07-11-2012 at 03:36 PM. Reason: Added interesting snippet
twobob is offline   Reply With Quote
Advert
Old 07-11-2012, 03:37 PM   #138
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by twobob View Post
well yes. but i want a unified format surely to parse.

Ill have look what my options are

EDIT: I'm looking.
And I noticed this: If your computer is not fast enough, you can speed up the compression at the expense of the compression ratio. You can use ’-me zero’ to speed up motion estimation, and ’-g 0’ to disable motion estimation completely (you have only I-frames, which means it is about as good as JPEG compression).

Which I thought I would mention.
I quit looking at the options when they were working "good enough". Thanks for the extra info that I may use.
geekmaster is offline   Reply With Quote
Old 07-13-2012, 01:01 AM   #139
cyclops0000
Padawan Learner
cyclops0000 has learned how to buy an e-book online
 
cyclops0000's Avatar
 
Posts: 33
Karma: 86
Join Date: Jul 2012
Location: Galactic Sector ZZ9 Plural Z Alpha
Device: Kindle Touch
Quote:
Originally Posted by geekmaster View Post
Yes. It would just be the Home key pressed and released twice during a short time period. You can read the keypress events in a script or a C program from the keyboard event device, or by calling the system "waitforkey" command. Of course, you need to do it in a separate thread so it does not stop video playback while checking for a keypress event. There are probably other ways to check for a keypress as well...
ok,i know its been a few days, but i just read your
post more thoroughly, and i saw what you said at the end about another way to check for a keypress event.
well i was fumbling around blindly while a video was going, trying to close the video(i had forgotten about ctr c) and i somehow typed in hd, which displays keypresses like command prompt in windows displays ctrl commands... with alot of arrows and a letter corresponding to the keypress.
i was wondering... could there be someway to use that to close videos?
btw sorry for grammatical/spelling errors, im typing this on my kindle... XD
cyclops0000 is offline   Reply With Quote
Old 07-13-2012, 01:27 AM   #140
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by cyclops0000 View Post
ok,i know its been a few days, but i just read your
post more thoroughly, and i saw what you said at the end about another way to check for a keypress event.
well i was fumbling around blindly while a video was going, trying to close the video(i had forgotten about ctr c) and i somehow typed in hd, which displays keypresses like command prompt in windows displays ctrl commands... with alot of arrows and a letter corresponding to the keypress.
i was wondering... could there be someway to use that to close videos?
btw sorry for grammatical/spelling errors, im typing this on my kindle... XD
"hd" is not on all the kindles. It is a shortcut for "hexdump -C" (hex dump with "sanitized" ASCII). It should not interrupt the video. Apparently with no parameters, it dumped STDIN (containing stuff that you type). I do not think that all hexdump versions default to STDIN, but it sounds like it may be useful (as long as it does not buffer STDIN and only output in "spurts"). All of the eink kindles that I tested have "watforkey" built in -- try it.

Last edited by geekmaster; 07-13-2012 at 01:31 AM.
geekmaster is offline   Reply With Quote
Advert
Old 07-13-2012, 11:29 PM   #141
twobob
( ͡° ͜ʖ ͡°){ʇlnɐɟ ƃǝs}Týr
twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.
 
twobob's Avatar
 
Posts: 6,586
Karma: 6299991
Join Date: Jun 2012
Location: uti gratia usura (Yao ying da ying; Mo ying da yieng)
Device: PW-WIFI|K5-3G+WIFI| K4|K3-3G|DXG|K2| Rooted Nook Touch
zcat santa.raw.gz |./Balsamic plughw:0 ./santa8kmono.wav

This doesn't actually work by the way. video is miles out of sync (I only scoop every 4 frames right now while I get it going) - also don't want the separate files - or separate inputs - or to run to mem - and various other bad things.

EDIT: I Added threading, The video is MILES out of sync but is pictures and sound.
And now not so jerky, Proof of utterly mangled concept, A true coding horror but with a little massage and some process unification I think this will get the job done.

the principle of seperate files is fundamentally flawed (as I knew it would be) but I wanted sound and vision and I got it. Time to tidy up the file handling and unify the data stream

Spoiler:
Code:
#include <sched.h>
#include <errno.h>
#include <getopt.h>
#include <asoundlib.h>
#include <math.h>
#include <pthread.h>
// HEAD WELD

//====================================================
// gmplay 1.5a - geekmaster's kindle video player
// Copyright (C) 2012 by geekmaster, with MIT license:
// http://www.opensource.org/licenses/mit-license.php
//----------------------------------------------------
// Tested on DX,DXG,K3,K4main,K4diags,K5main,K5diags.
//----------------------------------------------------
#include <sys/ioctl.h>     // ioctl
#include <sys/mman.h>     // mmap, munmap
#include <stdio.h>       // printf
#include <stdlib.h>     // malloc, free
#include <linux/fb.h>  // screeninfo
#include <sys/time.h> // gettimeofday
#include <unistd.h>  // usleep
#include <string.h> // memset, memcpy
#include <fcntl.h> // open, close, write
#include <time.h> // time
typedef unsigned long u64;
typedef unsigned int u32;
typedef unsigned short u16;
typedef unsigned char u8;
u32 __invalid_size_argument_for_IOC; // ioctl.h bug fix for tcc
//----- eink definitions from eink_fb.h and mxcfb.h -----
#define EU3 0x46dd
#define EU50 0x4040462e
#define EU51 0x4048462e
#define NUM_THREADS 3
struct update_area_t {int x1,y1,x2,y2,which_fx;u8 *buffer;};
struct mxcfb_rect {u32 top,left,width,height;};
struct mxcfb_alt_buffer_data {u32 phys_addr,width,height;
struct mxcfb_rect alt_update_region;};
struct mxcfb_update_data {struct mxcfb_rect update_region;
u32 waveform_mode,update_mode,update_marker;int temp;uint flags;
struct mxcfb_alt_buffer_data alt_buffer_data;};
struct mxcfb_update_data51 {struct mxcfb_rect update_region;
u32 waveform_mode,update_mode,update_marker;
u32 hist_bw_waveform_mode,hist_gray_waveform_mode;
int temp;uint flags;struct mxcfb_alt_buffer_data alt_buffer_data;};
//----- function prototypes -----
void gmplay4(void);
//void gmplay8(void);
int getmsec(void);
int gmlib(int);
//----- gmlib global vars -----
enum GMLIB_op {GMLIB_INIT,GMLIB_CLOSE,GMLIB_UPDATE,GMLIB_VSYNC};
u8 *fb0=NULL;     // framebuffer pointer
int fdFB=0;      // fb0 file descriptor
int teu=0;      // eink update time
u32 fs=0;      // fb0 stride
u32 MX=0;     // xres (visible)
u32 MY=0;    // yres (visible)
u32 VY=0;   // (VY>MY): mxcfb driver
u8 ppb=0;  // pixels per byte
u32 fc=0; // frame counter
#define FBSIZE (600/8*800)

//====================================
// gmlib - geekmaster function library
// op (init, update, vsync, close)
//------------------------------------
int gmlib(int op) {
	static struct update_area_t ua={0,0,600,800,21,NULL};
	static struct mxcfb_update_data ur={
			{0,0,600,800},257,0,1,0x1001,0,{0,0,0,{0,0,0,0}}};
	static struct mxcfb_update_data51 ur51={
			{0,0,600,800},257,0,1,0,0,0x1001,0,{0,0,0,{0,0,0,0}}};
	static int eupcode; static void *eupdata=NULL;
	struct fb_var_screeninfo screeninfo;
	if (GMLIB_INIT==op) { teu=getmsec(); fdFB=open("/dev/fb0",O_RDWR);
	ioctl(fdFB,FBIOGET_VSCREENINFO,&screeninfo);
	ppb=8/screeninfo.bits_per_pixel; fs=screeninfo.xres_virtual/ppb;
	VY=screeninfo.yres_virtual; MX=screeninfo.xres; MY=screeninfo.yres;
	ua.x2=MX; ua.y2=MY; ur.update_region.width=MX; ur.update_region.height=MY;
	fb0=(u8 *)mmap(0,MY*fs,PROT_READ|PROT_WRITE,MAP_SHARED,fdFB,0); // map fb0
	if (VY>MY) { eupcode=EU50; eupdata=&ur; ur.update_mode=0;
	if (ioctl(fdFB,eupcode,eupdata)<0) { eupcode=EU51; eupdata=&ur51; }
	} else { eupcode=EU3; eupdata=&ua; }
	system("eips -f -c;eips -c"); sleep(1);
	} else if (GMLIB_UPDATE==op) {
		if (ioctl(fdFB,eupcode,eupdata)<0) system("eips ''");  // 5.1.0 fallback
	} else if (GMLIB_VSYNC==op) { while (teu>getmsec()) usleep(1000); // fb0 busy
	} else if (GMLIB_CLOSE==op) { gmlib(GMLIB_UPDATE); sleep(1); // last screen
	system("eips -f -c;eips -c"); munmap(fb0,MY*fs); close(fdFB);
	} else { return -1; }
	return 0;
}
//====================================
// getmsec - get msec since first call
// (tick counter wraps every 12 days)
//------------------------------------
int getmsec(void) {
	int tc; static int ts=0; struct timeval tv;
	gettimeofday(&tv,NULL); tc=tv.tv_usec/1000+1000*(0xFFFFF&tv.tv_sec);
	if (0==ts) ts=tc;
	return tc-ts;
}

// Points to loaded WAVE file's data
unsigned char			*WavePtr;

// Size (in frames) of loaded WAVE file's data
snd_pcm_uframes_t		WaveSize;

// Sample rate
unsigned short			WaveRate;

// Bit resolution
unsigned char			WaveBits;

// Number of channels in the wave file
unsigned char			WaveChannels;

// For WAVE file loading
static const unsigned char Riff[4]	= { 'R', 'I', 'F', 'F' };
static const unsigned char Wave[4] = { 'W', 'A', 'V', 'E' };
static const unsigned char Fmt[4] = { 'f', 'm', 't', ' ' };
static const unsigned char Data[4] = { 'd', 'a', 't', 'a' };

snd_pcm_t *playback_handle;

unsigned int i,j;
int playcount;
int indexer;
int err;

//==================================
// gmplay4 - play video on 4-bit fb0
//----------------------------------
void gmplay4(void) {

	// We should scoop info from the unified file - and possibly via an indirect buffer.
	// Time indexing lookup - perhaps via a mutex provsioned RO value might be an idea
	// They are both TODO


	u32 i,x,y,b,p,off=(MY/2-400)*fs+MX/4-150,fbsize=FBSIZE; u8 fbt[FBSIZE];
	while (fread(fbt,fbsize,1,stdin)) { teu+=130; // teu: next update time
	if (getmsec()>teu+1000) continue; // drop frame if > 1 sec behind
	// gmlib(GMLIB_VSYNC); // wait for fb0 ready
	for (y=0;y<800;y++) for (x=0;x<600;x+=8) {
		b=fbt[600/8*y+x/8]; i=y*fs+x/2+off;
		p=(b&1)*240; b>>=1; fb0[i]=p|(b&1)*15; b>>=1;
		p=(b&1)*240; b>>=1; fb0[i+1]=p|(b&1)*15; b>>=1;
		p=(b&1)*240; b>>=1; fb0[i+2]=p|(b&1)*15; b>>=1;
		p=(b&1)*240; b>>=1; fb0[i+3]=p|(b&1)*15;
	} fc++; gmlib(GMLIB_UPDATE);

	//   printf("did this");
	}
}

int
playback_callback (snd_pcm_sframes_t nframes)
{

	playcount++;
	indexer = (4096*(playcount*2));


	if ((err = snd_pcm_writei (playback_handle, WavePtr+indexer, nframes)) < 0) {
		fprintf (stderr, "write failed (%s)\n", snd_strerror (err));
	}
	return err;

}

#pragma pack (1)
/////////////////////// WAVE File Stuff /////////////////////
// An IFF file header looks like this
typedef struct _FILE_head
{
	unsigned char	ID[4];
	// could be {'R', 'I', 'F', 'F'} or {'F', 'O', 'R', 'M'}
	unsigned int	Length;
	// Length of subsequent file (including remainder of header).
	//This is in Intel reverse byte order if RIFF, Motorola format if FORM.
	unsigned char	Type[4];
	// {'W', 'A', 'V', 'E'} or {'A', 'I', 'F', 'F'}
} FILE_head;


// An IFF chunk header looks like this
typedef struct _CHUNK_head
{
	unsigned char ID[4];	// 4 ascii chars that is the chunk ID
	unsigned int	Length;	// Length of subsequent data within this chunk. This is in Intel reverse byte
	// order if RIFF, Motorola format if FORM. Note: this doesn't include any
	// extra byte needed to pad the chunk out to an even size.
} CHUNK_head;

// WAVE fmt chunk
typedef struct _FORMAT {
	short				wFormatTag;
	unsigned short	wChannels;
	unsigned int	dwSamplesPerSec;
	unsigned int	dwAvgBytesPerSec;
	unsigned short	wBlockAlign;
	unsigned short	wBitsPerSample;
	// Note: there may be additional fields here, depending upon wFormatTag
} FORMAT;
#pragma pack()

/********************** compareID() *********************
 * Compares the passed ID str (ie, a ptr to 4 Ascii
 * bytes) with the ID at the passed ptr. Returns TRUE if
 * a match, FALSE if not.
 */

static unsigned char compareID(const unsigned char * id, unsigned char * ptr)
{
	register unsigned char i = 4;

	while (i--)
	{
		if ( *(id)++ != *(ptr)++ ) return(0);
	}
	return(1);
}

/********************** waveLoad() *********************
 * Loads a WAVE file.
 *
 * fn =			Filename to load.
 *
 * RETURNS: 0 if success, non-zero if not.
 *
 * NOTE: Sets the global "WavePtr" to an allocated buffer
 * containing the wave data, and "WaveSize" to the size
 * in sample points.
 */

// END WELD

static unsigned char waveLoad(const char *fn)
{
	const char				*message;
	FILE_head				head;
	register int			inHandle;

	if ((inHandle = open(fn, O_RDONLY)) == -1)
		message = "didn't open";

	// Read in IFF File header
	else
	{
		if (read(inHandle, &head, sizeof(FILE_head)) == sizeof(FILE_head))
		{
			// Is it a RIFF and WAVE?
			if (!compareID(&Riff[0], &head.ID[0]) || !compareID(&Wave[0], &head.Type[0]))
			{
				message = "is not a WAVE file";
				goto bad;
			}

			// Read in next chunk header
			while (read(inHandle, &head, sizeof(CHUNK_head)) == sizeof(CHUNK_head))
			{
				// ============================ Is it a fmt chunk? ===============================
				if (compareID(&Fmt[0], &head.ID[0]))
				{
					FORMAT	format;

					// Read in the remainder of chunk
					if (read(inHandle, &format.wFormatTag, sizeof(FORMAT)) != sizeof(FORMAT)) break;

					// Can't handle compressed WAVE files
					if (format.wFormatTag != 1)
					{
						message = "compressed WAVE not supported";
						goto bad;
					}

					WaveBits = (unsigned char)format.wBitsPerSample;
					WaveRate = (unsigned short)format.dwSamplesPerSec;
					WaveChannels = format.wChannels;
				}

				// ============================ Is it a data chunk? ===============================
				else if (compareID(&Data[0], &head.ID[0]))
				{
					// Size of wave data is head.Length. Allocate a buffer and read in the wave data
					if (!(WavePtr = (unsigned char *)malloc(head.Length)))
					{
						message = "won't fit in RAM";
						goto bad;
					}

					if (read(inHandle, WavePtr, head.Length) != head.Length)
					{
						free(WavePtr);
						break;
					}

					// Store size (in frames)
					WaveSize = (head.Length * 8) / ((unsigned int)WaveBits * (unsigned int)WaveChannels);

					close(inHandle);
					return(0);
				}

				// ============================ Skip this chunk ===============================
				else
				{
					if (head.Length & 1) ++head.Length;  // If odd, round it up to account for pad byte
					lseek(inHandle, head.Length, SEEK_CUR);
				}
			}
		}

		message = "is a bad WAVE file";
		bad:	close(inHandle);
	}

	printf("%s %s\n", fn, message);
	return(1);
}


/*********************** free_wave_data() *********************
 * Frees any wave data we loaded.
 *
 * NOTE: A pointer to the wave data be in the global
 * "WavePtr".
 */

static void free_wave_data(void)
{
	if (WavePtr) free(WavePtr);
	WavePtr = 0;
}


// Showtime

main (int argc, char *argv[])
{

	int i;
	gmlib(GMLIB_INIT);

	char *wavename = argv[2];

	if (!waveLoad(wavename))
	{
		register int		err;
	}
	else
	{

		printf("Loaded: %s\n", wavename);

	}

	snd_pcm_hw_params_t *hw_params;
	snd_pcm_sw_params_t *sw_params;
	snd_pcm_sframes_t frames_to_deliver;
	int nfds;
	int err;
	struct pollfd *pfds;

	if ((err = snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0)) < 0) {
		fprintf (stderr, "cannot open audio device %s (%s)\n",
				argv[1],
				snd_strerror (err));
		exit (1);
	}

	if ((err = snd_pcm_set_params(playback_handle,
			SND_PCM_FORMAT_S16_LE,
			SND_PCM_ACCESS_RW_INTERLEAVED,
			1,
			8000,
			0,
			900000)) < 0) {   /* 0.5sec */
		printf("Playback open error: %s\n", snd_strerror(err));
		exit(EXIT_FAILURE);
	}

	/* tell ALSA to wake us up whenever 4096 or more frames
		   of playback data can be delivered. Also, tell
		   ALSA that we'll start the device ourselves.
	 */

	if ((err = snd_pcm_sw_params_malloc (&sw_params)) < 0) {
		fprintf (stderr, "cannot allocate software parameters structure (%s)\n",
				snd_strerror (err));
		exit (1);
	}
	if ((err = snd_pcm_sw_params_current (playback_handle, sw_params)) < 0) {
		fprintf (stderr, "cannot initialize software parameters structure (%s)\n",
				snd_strerror (err));
		exit (1);
	}
	if ((err = snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096)) < 0) {
		fprintf (stderr, "cannot set minimum available count (%s)\n",
				snd_strerror (err));
		exit (1);
	}
	if ((err = snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 0U)) < 0) {
		fprintf (stderr, "cannot set start mode (%s)\n",
				snd_strerror (err));
		exit (1);
	}
	if ((err = snd_pcm_sw_params (playback_handle, sw_params)) < 0) {
		fprintf (stderr, "cannot set software parameters (%s)\n",
				snd_strerror (err));
		exit (1);
	}

	/* the interface will interrupt the kernel every 4096 frames, and ALSA
		   will wake up this program very soon after that.
	 */

	if ((err = snd_pcm_prepare (playback_handle)) < 0) {
		fprintf (stderr, "cannot prepare audio interface for use (%s)\n",
				snd_strerror (err));
		exit (1);
	}


	// SETUP THE SCREEN

	pthread_t threads[NUM_THREADS];
	int rc;
	long t;

	gmlib(GMLIB_INIT);

	// Make a thread


	printf("In main: creating thread %ld\n", t);
	char *message1 = "Video Thread";

	// This is the thread.

	rc = pthread_create(&threads[t], NULL, gmplay4, (void*) message1);

	if (rc){
		printf("ERROR; return code from pthread_create() is %d\n", rc);
		exit(-1);
		i=getmsec()/100; printf("%d frames in %0.1f secs = %2.1f FPS\n",
				fc,(double)i/10.0,(double)fc*10.0/i);
		gmlib(GMLIB_CLOSE);

	}


	while (1) {

		// HERE IS OUR UPDATE LOOP SEED

		// We should create a buffer area to scoop audio into from the main feed -
		// as audio is bloody picky about actually playing
		// it would be nice to consider some type of time indexing for this I guess to give us a sense of info unity.


		/* wait till the interface is ready for data, or 1 second
			   has elapsed.
		 */

		if ((err = snd_pcm_wait (playback_handle, 1000)) < 0) {
			fprintf (stderr, "poll failed (%s)\n", strerror (errno));
			break;
		}

		/* find out how much space is available for playback data */

		if ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) < 0) {
			if (frames_to_deliver == -EPIPE) {
				fprintf (stderr, "an xrun occured\n");
				break;
			} else {
				fprintf (stderr, "unknown ALSA avail update return value (%d)\n",
						frames_to_deliver);
				break;
			}
		}

		if (	frames_to_deliver > 4096 ){ frames_to_deliver = 4096;}

		/* deliver the data */

		if (playback_callback (frames_to_deliver) != frames_to_deliver) {
			fprintf (stderr, "playback callback failed\n");
			break;
		}
	}
	i=getmsec()/100; printf("%d frames in %0.1f secs = %2.1f FPS\n",
			fc,(double)i/10.0,(double)fc*10.0/i);
	gmlib(GMLIB_CLOSE);
	snd_pcm_close (playback_handle);
	exit (0);
}


Tomorrows job - make this actually work at all heheheh still sound and video together is always good, however mangled and wrong

It's the future.

Last edited by twobob; 07-14-2012 at 10:37 AM. Reason: added call structure, and some sense - and partial success
twobob is offline   Reply With Quote
Old 07-14-2012, 12:07 AM   #142
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
I think we should skip the vsync call and sync to the audio stream instead.

On a K3, it spends all its time in the eink update. The vsync does not really do anything on a K3 because it is set for 7FPS and the eink call returns at 7FPS average (+/- a little). Sometimes it waits a tiny amount of time, and sometimes it is a little late (compensated on next frame by next teu+=130.

The vsync is really there for the K4/K5 that have asynchronous eink updates (which return WAY before the eink is ready). You really must wait awhile on them before touching the framebuffer.

As I see it, we need two threads that each spend virtually all their time inside the update call (sound or eink). For any interactivity (scanning for keyboard or touchscreen events) we need a third GUI thread too.

Doing it with callbacks cannot work while the thread is almost ALWAYS stalled inside the eink update on a K3.

pthreads is not all that difficult. Even the ALSA docs say you need multiple threads (just like I suggested above).

And with threads, we can just use the tiny sound support I used in my "noisy rhythm" demo. With interleaved raw A/V input (compressed with gzip or other compression) we do not need all that "visual space hogging" AVI support.

IMHO.

Need sleep now. Will try adding threads and stripping out the fluff tomorrow. Tomorrow I will pick up where you left off here (or in a PM, whichever is newer).

After it works, I will add simple decompression inside this code so we do not need to pipe from zcat.

Or you do it. Whichever gets there first. Or I might add audio to my sound demo. I will sleep on it. Cannot make decisions now. TTYL...

Last edited by geekmaster; 07-14-2012 at 07:54 AM.
geekmaster is offline   Reply With Quote
Old 07-14-2012, 07:21 AM   #143
twobob
( ͡° ͜ʖ ͡°){ʇlnɐɟ ƃǝs}Týr
twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.twobob ought to be getting tired of karma fortunes by now.
 
twobob's Avatar
 
Posts: 6,586
Karma: 6299991
Join Date: Jun 2012
Location: uti gratia usura (Yao ying da ying; Mo ying da yieng)
Device: PW-WIFI|K5-3G+WIFI| K4|K3-3G|DXG|K2| Rooted Nook Touch
Right. Vsync. Threads. Yep this was never about showing it working. Simply about making sound and vision together before I got in bed.

: ) Thanks for your great and as always sage feedback.

EDIT: Updated the above code and forwarded via PM.

Running this code now WITHOUT a stdin stream produces less than 1% use on K3

TODO: Unified format - striped in some way to be determined.

Settle on an audio format (or make it dynamically determine format of audio (or fix this) / channel count / rate etc. and setup the buffer skips accordingly.)

Look into Timestamped/timeindexed data stream and maybe locking via mtuex all protected stream info

Replace stdin and Zcat Zcat uses about 7% of sys resources at the moment. that will be a nice saving assuming the internal de/compression aces that for performance.

Try threading audio rather than callback (although audio is now pretty solid), add some thread efficiency monitoring code.

Look into suspending audio buffers and providing << >> |> [] ["] Transport options.

Add a GUI. Nice to have

Last edited by twobob; 07-14-2012 at 10:58 AM. Reason: added that I updated the code, added TODO list
twobob is offline   Reply With Quote
Old 07-15-2012, 01:01 AM   #144
cyclops0000
Padawan Learner
cyclops0000 has learned how to buy an e-book online
 
cyclops0000's Avatar
 
Posts: 33
Karma: 86
Join Date: Jul 2012
Location: Galactic Sector ZZ9 Plural Z Alpha
Device: Kindle Touch
so would the waitforkey command or whatever i end up using to shutdoen the player, how would i implement it into the kindle? would i ave to repack the videoplayer or something, or would i just need to add a command to execute the script when i launch the videoplayer?
cyclops0000 is offline   Reply With Quote
Old 07-15-2012, 01:16 AM   #145
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by cyclops0000 View Post
so would the waitforkey command or whatever i end up using to shutdown the player, how would i implement it into the kindle? would i ave to repack the videoplayer or something, or would i just need to add a command to execute the script when i launch the videoplayer?
You can use "waitforkey" in a script, but it stalls waiting for input. I ran it (in a script) in the background by appending an "&" (to the script) and piped waitforkey output "> /tmp/keypress". It ran in a loop so it would restart after storing a keycode.The foreground script would read /tmp/keypress with "cat /tmp/keypress" and then after seeing a keypress write a "0 0" for "no keypress" back to the file. It would periodically check that file during its main run loop (which also contained a sleep call and only checked for keypresses about 10 times per second). Yes a hack, but it worked well for my needs. I am sure there is a better way, but sometimes getting a hack working quickly is better than giving up in frustration while trying to figure out "the RIGHT way" to do something.

You can also process keypresses by reading the binary /dev/event* device data for the keyboard (using hexdump from a script, or in a C program).

There are other ways to, but they would require more research.

Last edited by geekmaster; 07-15-2012 at 09:20 AM.
geekmaster is offline   Reply With Quote
Old 07-15-2012, 03:43 AM   #146
peter12345
Connoisseur
peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.peter12345 ought to be getting tired of karma fortunes by now.
 
peter12345's Avatar
 
Posts: 93
Karma: 498332
Join Date: Jun 2012
Device: 2 * K3 with 3G, Sony PRS-T1
Fascinating, would have thought that such a thing was impossible?
peter12345 is offline   Reply With Quote
Old 07-15-2012, 09:22 AM   #147
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by peter12345 View Post
Fascinating, would have thought that such a thing was impossible?
Things are only "impossible" if you give up trying new ways to solve them before you find a working solution. Doing the "impossible" takes a LOT of persistence (and creativity) in many cases. Much of my life has been defined by successfully doing things that others claimed was "impossible" (which I took as a challenge). I like seeing people (literally) "drop their jaws" when I show them my stuff.

Last edited by geekmaster; 07-16-2012 at 08:38 AM.
geekmaster is offline   Reply With Quote
Old 07-16-2012, 06:25 PM   #148
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
I have a gmplay-2.0, which plays video in one thread and "noisy rhythms" audio in another thread. On my k5, it plays perfectly, using abou 12% CPU. Of course, knowing that the K3 eink update burns CPU, I need to test it there too... EDIT: On the K3, it burns 93% CPU (mostly in the eink update call). But the video plays full speed even with synthesized sound. EDIT2: After recompiling with optimization (-O3) it now uses 82% CPU on the K3 -- a useful improvement.

Old message:
Spoiler:
Strangely, on the K3, my original noisy demo has the sound fading in and out. It was working perfectly before. Strange change in behavior. Only difference is I use -O3 when compiling it this time. Restarting it now to try again on the K3.

EDIT: Restart did not help K3. The "noisy rhythms" demo drops in and out even compiled without -O3. It was working before on the DX/K3/K5. The K5 still works.

Strange...

I just tested twobob's tones demo, and it has the same "in and out" behavior. The dropouts are abrupt, but it resume with a fade-in (just like when you connect headphones for other audio). The interruptions are non-periodic (about 1 second in, 1 second out, varying by about 1/2 second in duration). It is a general sound problem affecting tones and noise demos on my k3, not fixed by a restart.

I *was* messing with amixer and alsamixer yesterday. I wonder if one of them wrote crap config data to /etc or something...

UPDATE: My K3 sound works (for noisy and tones) after turning down the volume. It must have some kind of overload detection that makes it drop out for a bit if it is too loud. I had it turned up for the quiet 8kb8mono.wav|aplay test (which is still quite even at max volume).

So, if there are sound dropouts, turn down the volume.

The next step in this path of combined Audio/Video evolution of gmplay is to use sound interleaved into the video file, instead of synthesizing it.

Last edited by geekmaster; 07-16-2012 at 06:43 PM.
geekmaster is offline   Reply With Quote
Old 07-16-2012, 07:52 PM   #149
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Arrow Movies with sound, coming soon to a kindle near you!

This is a "Work In Progress". It is gmplay 2.0 (now with sound). This is just a demo showing multithreaded sound and video. The main thread is where future GUI stuff will go. The video thread exits after the video is done playing. The audio continues indefinately, until killed when the main program exits.

The sound in this demo is synthesized, as described in the "noisy rhythms" demo in the "geekmaster's sound demos" thread).

The next step is to add sound to the .gmv.gz movie files, and replace synthesized sound with recorded sound.

On a K3, if you get periodic sound dropouts, turn down your sound a bit with the Vol- button.
PHP Code:
//====================================================
// gmplay 2.0 - geekmaster's kindle video player
// Copyright (C) 2012 by geekmaster, with MIT license:
// http://www.opensource.org/licenses/mit-license.php
// This version has algorithmically synthesized sound!
//----------------------------------------------------
// Tested on DX,DXG,K3,K4main,K4diags,K5main,K5diags.
//----------------------------------------------------
#include <pthread.h>         // pthread_
#include <asoundlib.h>      // snd_pcm_
#include <sys/ioctl.h>     // ioctl
#include <sys/mman.h>     // mmap, munmap
#include <stdio.h>       // printf
#include <stdlib.h>     // malloc, free
#include <linux/fb.h>  // screeninfo
#include <sys/time.h> // gettimeofday
#include <unistd.h>  // usleep
#include <string.h> // memset, memcpy
#include <fcntl.h> // open, close, write
#include <time.h> // time
typedef unsigned long u64;
typedef unsigned int u32;
typedef unsigned short u16;
typedef unsigned char u8;
u32 __invalid_size_argument_for_IOC// ioctl.h bug fix for tcc
//----- eink definitions from eink_fb.h and mxcfb.h -----
#define EU3 0x46dd
#define EU50 0x4040462e
#define EU51 0x4048462e
struct update_area_t {int x1,y1,x2,y2,which_fx;u8 *buffer;};
struct mxcfb_rect {u32 top,left,width,height;};
struct mxcfb_alt_buffer_data {u32 phys_addr,width,height;
    
struct mxcfb_rect alt_update_region;};
struct mxcfb_update_data {struct mxcfb_rect update_region;
    
u32 waveform_mode,update_mode,update_marker;int temp;uint flags;
    
struct mxcfb_alt_buffer_data alt_buffer_data;};
struct mxcfb_update_data51 {struct mxcfb_rect update_region;
   
u32 waveform_mode,update_mode,update_marker;
   
u32 hist_bw_waveform_mode,hist_gray_waveform_mode;
   
int temp;uint flags;struct mxcfb_alt_buffer_data alt_buffer_data;};
//----- function prototypes -----
void sndplay(void);
void gmplay4(void);
void gmplay8(void);
int getmsec(void);
int gmlib(int);
//----- gmlib global vars -----
enum GMLIB_op {GMLIB_INIT,GMLIB_CLOSE,GMLIB_UPDATE,GMLIB_VSYNC};
snd_pcm_t *hp;     // sound play handle
u8 *fb0=NULL;     // framebuffer pointer
int fdFB=0;      // fb0 file descriptor
int teu=0;      // eink update time
u32 fs=0;      // fb0 stride
u32 MX=0;     // xres (visible)
u32 MY=0;    // yres (visible)
u32 VY=0;   // (VY>MY): mxcfb driver
u8 ppb=0;  // pixels per byte
u32 fc=0// frame counter
#define FBSIZE (600/8*800)
//============================
// sndplay - play sound buffer
// this plays until we kill it
//----------------------------
void sndplay(void) {
    
unsigned char buf[1024]; // play buffer
    
snd_pcm_sframes_t fc;   // frame count
    
unsigned int i,j;
    for (
i=0;;i++) { // forever
        
for (j=0;j<sizeof(buf);j++) buf[j]=(random()&3)*i*350;
        
fc=snd_pcm_writei(hp,buf,sizeof(buf));  // play buffer
        
if (fc<0fc=snd_pcm_recover(hp,fc,0); // error recovery
    
}
}  
//==================================
// gmplay4 - play video on 4-bit fb0
//----------------------------------
void gmplay4(void) {
    
u32 i,x,y,b,p,off=(MY/2-400)*fs+MX/4-150,fbsize=FBSIZEu8 fbt[FBSIZE];
    while (
fread(fbt,fbsize,1,stdin)) { teu+=130// teu: next update time
        
if (getmsec()>teu+1000) continue; // drop frame if > 1 sec behind
        
gmlib(GMLIB_VSYNC); // wait for fb0 ready
        
for (y=0;y<800;y++) for (x=0;x<600;x+=8) {
            
b=fbt[600/8*y+x/8]; i=y*fs+x/2+off;
            
p=(b&1)*240b>>=1fb0[i]=p|(b&1)*15b>>=1;
            
p=(b&1)*240b>>=1fb0[i+1]=p|(b&1)*15b>>=1;
            
p=(b&1)*240b>>=1fb0[i+2]=p|(b&1)*15b>>=1;
            
p=(b&1)*240b>>=1fb0[i+3]=p|(b&1)*15;
        } 
fc++; gmlib(GMLIB_UPDATE);
    }
}
//==================================
// gmplay8 - play video on 8-bit fb0
//----------------------------------
void gmplay8(void) {
    
u32 i,x,y,b,fbsize=FBSIZEu8 fbt[FBSIZE];
    while (
fread(fbt,fbsize,1,stdin)) { teu+=130// teu: next update time
        
if (getmsec()>teu+1000) continue; // drop frame if > 1 sec behind
        
gmlib(GMLIB_VSYNC); // wait for fb0 ready
        
for (y=0;y<800;y++) for (x=0;x<600;x+=8) {
            
b=fbt[600/8*y+x/8]; i=y*fs+x;
            
fb0[i]=(b&1)*255b>>=1fb0[i+1]=(b&1)*255b>>=1;
            
fb0[i+2]=(b&1)*255b>>=1fb0[i+3]=(b&1)*255b>>=1;
            
fb0[i+4]=(b&1)*255b>>=1fb0[i+5]=(b&1)*255b>>=1;
            
fb0[i+6]=(b&1)*255b>>=1fb0[i+7]=(b&1)*255;
        } 
fc++; gmlib(GMLIB_UPDATE);
    }
}
//====================================
// gmlib - geekmaster function library
// op (init, update, vsync, close)
//------------------------------------
int gmlib(int op) {
    static 
struct update_area_t ua={0,0,600,800,21,NULL};
    static 
struct mxcfb_update_data ur={
        {
0,0,600,800},257,0,1,0x1001,0,{0,0,0,{0,0,0,0}}};
    static 
struct mxcfb_update_data51 ur51={
        {
0,0,600,800},257,0,1,0,0,0x1001,0,{0,0,0,{0,0,0,0}}};
    static 
int eupcode; static void *eupdata=NULL;
    
struct fb_var_screeninfo screeninfo;
    if (
GMLIB_INIT==op) { teu=getmsec(); fdFB=open("/dev/fb0",O_RDWR);
        
snd_pcm_open(&hp,"default",SND_PCM_STREAM_PLAYBACK,0);
        
snd_pcm_set_params(hp,SND_PCM_FORMAT_U8,
            
SND_PCM_ACCESS_RW_INTERLEAVED,1,8000,1,500000); // 0.5 sec latency
        
ioctl(fdFB,FBIOGET_VSCREENINFO,&screeninfo);
        
ppb=8/screeninfo.bits_per_pixelfs=screeninfo.xres_virtual/ppb;
        
VY=screeninfo.yres_virtualMX=screeninfo.xresMY=screeninfo.yres;
        
ua.x2=MXua.y2=MYur.update_region.width=MXur.update_region.height=MY;
        
fb0=(u8 *)mmap(0,MY*fs,PROT_READ|PROT_WRITE,MAP_SHARED,fdFB,0); // map fb0
        
if (VY>MY) { eupcode=EU50eupdata=&urur.update_mode=0;
            if (
ioctl(fdFB,eupcode,eupdata)<0) { eupcode=EU51eupdata=&ur51; }
        } else { 
eupcode=EU3eupdata=&ua; }
        
system("eips -f -c;eips -c"); sleep(1);
    } else if (
GMLIB_UPDATE==op) {
        if (
ioctl(fdFB,eupcode,eupdata)<0system("eips ''");  // 5.1.0 fallback
    
} else if (GMLIB_VSYNC==op) { while (teu>getmsec()) usleep(1000); // fb0 busy
    
} else if (GMLIB_CLOSE==op) { gmlib(GMLIB_UPDATE); sleep(1); // last screen
        
system("eips -f -c;eips -c"); munmap(fb0,MY*fs); close(fdFB);
        
snd_pcm_close(hp);
    } else { return -
1; }
    return 
0;
}
//====================================
// getmsec - get msec since first call
// (tick counter wraps every 12 days)
//------------------------------------
int getmsec(void) {
    
int tc; static int ts=0struct timeval tv;
    
gettimeofday(&tv,NULL); tc=tv.tv_usec/1000+1000*(0xFFFFF&tv.tv_sec);
    if (
0==tsts=tc;
    return 
tc-ts;
}
//==================
// main - start here
//------------------
int main(void) {
    
pthread_t athread,vthreadint i;
    
gmlib(GMLIB_INIT);
    
pthread_create(&athread,NULL,(void *)sndplay,NULL);
    
pthread_create(&vthread,NULL,(void *)(ppb-1?gmplay4:gmplay8),NULL);
    
pthread_join(vthread,NULL); // wait vthread done
    
i=getmsec()/100printf("%d frames in %0.1f secs = %2.1f FPS\n",
        
fc,(double)i/10.0,(double)fc*10.0/i);
    
gmlib(GMLIB_CLOSE);
    return 
0;


Last edited by geekmaster; 07-17-2012 at 10:31 PM.
geekmaster is offline   Reply With Quote
Old 07-16-2012, 09:58 PM   #150
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Arrow kindle audio/video gmv format

Below is a detailed description about how I plan to add sound into my GMV (geekmaster video) file format, in a (mostly) backward-compatible way. Also described is a little bit about how to process that data to overcome potential problems caused by eink update time variability. Click the show button if that sort of thing interests you.

Technical Details:
Spoiler:
I think I have mostly decided what format to use for adding sound to GMV files. Now they are raw dithered 800x600 bit-packed video frames with no metadata (60,000 bytes/frame). To add sound, I insert 60,000-byte audio frames as needed. If a GMV file contains audio, then the first frame will be an audio frame. Audio frames will contain a frame header with a unique signature (magic number) and WAVE style sound metadata. Because each audio frame has its own metadata, we can switch encoding methods for each audio frame. This will allow using A-Law for simple encoding, or raw 16-bit sound, or celp/speex, celt, or mp3 codecs, depending on the content at THAT POINT in the video. Would could have speex for a lecture and mp3 for intro music IN THE SAME VIDEO.

The audio metadata will also say how many frames until the next audio frame, so a simple player only needs to count frames (no need to search for metadata signatures).

The player will buffer 1MB of data before starting to play, so it can synchronize the sound and video together.

The complication is that gmplay uses 130 msec frames because that is the best AVERAGE framerate that a K3 can do. Any faster and complex scenes get nasty artifacts. Any slower and the animation gets too jerky. But that is only an AVERAGE. On the K3, the eink update does not return until the framebuffer is ready to be written with new data (130 msec, more or less). On the K4/K5, the eink update returns almost immediately and we wait the rest of the 130 msec frame time before writing to the framebuffer, so the videos play at the same average speed on all kindles.

For simple scenes, the K3 can return much faster than 130 msec, but for complex scene changes, it can take up to 300 msec to return (such as when cutting from a dark scene to a bright scene). Sometimes, mutiple scenes changes within 1 second can cause the K3 to fall up to one second behind (and occasionally more in some videos). If it falls more than one second behind, gmplay drops frames to keep it from falling more than one second behind. When the video gets back to simple scene changes, the K3 can catch back up to "real time".

If you start framedropping before one second delay, most videos can get too many frame drops, or can even get into a state where every-other frame gets dropped resulting in very jerky motion.

Because the eink updates are the limiting factor and we need to allow for up to one second occasional delays in the video, getting the audio to stay in sync adds significant complication. Because framedropping makes the video play time almost exacly match the audio play time, we can just ignore long eink updates and the following dropped frames and let the sound occasionally get up to one second AHEAD of the video (resynchronizing after framedrops), or we can make every audio frame match the video frame just written to the framebuffer (which would require sometimes skipping or repeating one or more audio buffers. The key to making that less annoying is to make the audio buffers play for exactly 130 msec as well, and pack multiple audio buffers into each audio frame. Using variable-compression audio formats would complicate this.

Because we can use predictable A-Law logarithmic fixed compression on the audio, we can use 8000 bytes/sec audio (which is 7 seconds of audio per 60,000 byte audio frame, with room to spare). We only need to insert an audio frame every 7 seconds. We can break that up into 130 msec buffers, to match the video frames. We can play the audio buffer that matches the current video frame. If an audio buffer completes before the next video frame is ready, we can REPLAY the existing audio frame (during complex scene changes). Hopefully, this will be during silent cut scenes or violent noisy action scenes where such audio glitches may go unnoticed. When the video is "catching up" with framedrops, we can also drop those audio frames.

There are more complex ways of changing the audio speed using three-phase ring buffers (with single phase fades across buffer I/O crossover), but we will try the simpler way first.


EDIT: It just occurred to me that it may be simpler to let the audio run without interruption and sync the video to it. In other words, whenever the eink update returns, it just plays whatever video frame is the closest match to the audio, even if that means dropping multiple frames to get back in sync. The simplest code may be to let the AUDIO thread advance the NEXT VIDEO frame counter. Then the video can just use that frame counter (or pointer), instead of dropping frames by itself. That would guarantee sound sync, without any audible hiccups. It makes the previous idea seem complicated. Things get simpler if you just think about them enough...

After the GMV files have embedded sound, I will change the gmplay 2.0 code you can see in the previous post so that it can play this new audio sound track on the kindle speakers (or headphones) while simultaneously playing the video on the kindle eink screen.

Last edited by geekmaster; 07-17-2012 at 09:44 PM.
geekmaster is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Hacks free kindle video player geekmaster Amazon Kindle 0 05-07-2012 12:17 AM
Video Player wilbrich Kobo Tablets 1 11-24-2011 02:40 PM
Video player, Video streaming,Remote mint121 enTourage eDGe 1 11-18-2011 03:31 PM
Frustrated with the video player TonyToews Kobo Tablets 10 11-01-2011 01:15 PM


All times are GMT -4. The time now is 06:27 AM.


MobileRead.com is a privately owned, operated and funded community.