Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > More E-Book Readers > iRex > iRex Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 12-11-2006, 06:23 PM   #1
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
Lightbulb Fixing iPdf for iRex

I believe I have finally figured out the ipdf sharpness and image quality issue, without needing to modify Poppler!

Originally iRex was taking 24 bit bitmaps from Poppler and doing something interesting with them. I never realized how interesting until this past weekend. They were doing a color space cube reduction and letting the cube reduction kinda sorta correct them into 4 bit per pixel gray.

I say interesting because conversion from RGB to 8 bit grey scale is essentially error free to the human eye, there is no need to use a color cube reduction. In addition their color cube reduction wasn't doing anything intelligent vis-a-vis the human eye. The human eye is far more sensitive to green than either or red or blue. iRex though did a simple linear color cube reduction.

So why did iRex say the ipdf converted to have Poppler serve up Mono 8 was inferior in image quality for pictures?

While Mono 8 is good enough, the iLiad is a Mono 4 device. The iLiad's frame buffer (and my code as well) was not doing anything smart in converting Mono 8 to Mono 4, so in some cases you'd get some rather nasty posterization in images.

The code below takes Poppler's Mono 8 and crunches it to Mono 4 with FS dithering for the error distribution. This more directly addresses what the cube reduction was doing by reducing the number of colors in the Poppler image to indirectly reduce the number of grays in the image.

But what about the glyphs? We still want our razor sharp glyphs right? Didn't the FS error distribution do bad things to the glyphs?

Technically, white on the iLiad is actually 239, but iRex tells Poppler to draw text on 255 paper. This use of 255 rather than 239 causes their cube routine to distribute an error because there is no 255 on the iLiad. This distributed white error would in essence chew the edges of glyphs and make them appear "moth eaten" as little white errors were distributed into them.

The code below understands 255 is "white" and exempts it from error distribution. I tried just telling Poppler to draw on 239 paper but Paul's scanned books (and others) are drawn on 255 paper as well...

This code drops into PDFCore.cpp. Don't forget to also tweak that same file to use Mono 8 mode with Poppler.

Code:
    // check result
    if (ret == Render_Error || ret == Render_Invalid)
    {
        PV_ERRORPRINTF("Render page error!");
        return ret;
    }

    // bitmap has been allocated
    info.bitmap = outputDev->takeBitmap();

    // Convert Mono 8 to Mono 4
    int srcWidth = info.bitmap->getWidth();
    int srcHeight = info.bitmap->getHeight();
    char *src = (char *)info.bitmap->getDataPtr();

    int origGrey = 0;
    int grey = 0;
    int error = 0;
    int *errors = (int *)alloca((srcWidth + 2) * 4);
    memset(errors, 0, (srcWidth + 2) * 4);
    int errorRight = 0;
    int errDownRight = 0;

    for (int y = 0; y < srcHeight; y++) {
      for (int x = 0; x < srcWidth; x++) {
        // Get Mono 8 pixel and add error distribution
        origGrey = *src + errorRight + errors[x+1];

        // Convert pixel to Mono 4, clip < 0 and > Mono 4
        if (origGrey <= 0) {
          grey = 0;
        } else if (origGrey  >= 0x00ff) {
          grey = 0x00ff; // No error for white of 255
        } else {
          grey = origGrey & 0x00f0;
        }

        // Put Mono 4 pixel with distributed error back
        *src++ = grey;

        // Calculate error
        error = origGrey - grey;

        // Distribute error
        errorRight = (error * 7) >> 4;
        errors[x] += (error * 3) >> 4;
        errors[x+1] = ((error * 5) >> 4) + errDownRight;
        errDownRight = error >> 4;
      }
    }

    if (ret == Render_Done)
    {
#if (PV_PROFILE_ON)
scotty1024 is offline   Reply With Quote
Old 12-11-2006, 06:35 PM   #2
NatCh
Gizmologist
NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.NatCh ought to be getting tired of karma fortunes by now.
 
NatCh's Avatar
 
Posts: 11,605
Karma: 926222
Join Date: Jan 2006
Location: Republic of Texas Embassy at Jackson, TN
Device: Nook STGR
Ho, Scotty1024! Welcome back! We've been worried about you!
NatCh is offline   Reply With Quote
 
Enthusiast
Old 12-11-2006, 06:39 PM   #3
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
I never posted the sources of ipdf's 4 and 5 because I thought iRex and I would be able to work out some amicable agreement.

That never happened and I've pretty much given up hope that it ever will.

They recently released 2.8 and they attempted to implement my ipdf4 trick of writing directly to the /dev/fb as a memory mapped buffer.

They didn't do a very efficient job of it and it appears to me that they also introduced a bug in the process.

First the bug.

If you memory map something, you need to un-map it.

Code:
munmap(screenMem, screenSize);
They never un-map it: oops.

I could have sworn I sent M the code below, they must have lost it...

In any case, their version of it in 2.8 touches nearly every pixel to be drawn twice. That makes the code below nearly twice as quick at putting the image up on the display since it doesn't.

Code:
void XMgr::drawImage(SplashBitmap * bmp, 
                     int xSrc , int ySrc , 
                     int xDest, int yDest) 
{
    // Get our source image width and calculate delta frem screen width
    int srcWidth = bmp->getWidth();
    int deltaX = SCREEN_WIDTH - srcWidth;
    if (deltaX < 0) {
      // Source too wide, trim to screen width
      srcWidth = SCREEN_WIDTH;
    }

    // Get our source image height and calculate delta frem screen height
    int srcHeight = bmp->getHeight();
    int deltaY = CLIENT_AREA - srcHeight;
    if (deltaY < 0) {
      // Source too tall, trim to screen height
      srcHeight = CLIENT_AREA;
    }

    // Now move image data into shared memory buffer
    char *src = (char*)bmp->getDataPtr();
    char *dest = screenMem;
    for (int i = 0; i < srcHeight; i++) {
      // Copy source line of data
      memcpy(dest, src, srcWidth);
      src += srcWidth;
      dest += srcWidth;

      // Fill rest of line with white
      if (deltaX > 0) {
        memset(dest, 0xf0, deltaX);
        dest += deltaX;
      } else {
        // Skip extra source line material
        src -= deltaX;
      }
    }

    // Fill rest of page with white
    if (deltaY > 0) {
      memset(dest, 0xf0, deltaY * SCREEN_WIDTH);
    }
My code is also slightly better documented than theirs. Even though they know they are posting their code as "reference material" they still haven't done much to enhance their usage of comments.
scotty1024 is offline   Reply With Quote
Old 12-11-2006, 07:16 PM   #4
CommanderROR
eink fanatic
CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.CommanderROR is fluent in JavaScript as well as Klingon.
 
CommanderROR's Avatar
 
Posts: 2,022
Karma: 4924
Join Date: Mar 2006
Location: Germany
Device: STAReBOOK, iRex Iliad, Sony 505, Kindle 2
Thanks for coming back Scotty1024.

I'm looking forward to seeing a news IPDF6 or something like it with a nice package done by Yokos...so that all the noobs like me can safely install it on 2.8...:-)

Are you going to continue your work on the Iliad Software or are you going to quit due to iRexs' uncooperative attitude?
CommanderROR is offline   Reply With Quote
Old 12-12-2006, 04:46 AM   #5
kapoira
Connoisseur
kapoira has a complete set of Star Wars action figures.kapoira has a complete set of Star Wars action figures.kapoira has a complete set of Star Wars action figures.
 
Posts: 81
Karma: 292
Join Date: Nov 2006
Device: i62HD + T68
thanks scotty i was pasing images to a pdf, and i was thinking that the background of the image wasn't pure white because the convert of the images to grayscale (i see that the borders of the iliad are more white), and i was thinking that the error was because to use a grayscale pdf in the ipdf of the iliad, now you have showed to me the problem was the ipdf no the pdf, thanks, thanks, and sorry for this next question,

you will put a new ipdf compiled with this problem resolved, thanks for the response if it yes i will wait if it's not i will try to compile it ;-)
kapoira is offline   Reply With Quote
Old 12-12-2006, 05:54 AM   #6
jęd
Evangelist
jęd has a complete set of Star Wars action figures.jęd has a complete set of Star Wars action figures.jęd has a complete set of Star Wars action figures.
 
Posts: 458
Karma: 293
Join Date: May 2006
Quote:
Originally Posted by scotty1024
I never posted the sources of ipdf's 4 and 5 because I thought iRex and I would be able to work out some amicable agreement.

That never happened and I've pretty much given up hope that it ever will.
The best time to ask for cash is usually before you've opened the IDE...!
jęd is offline   Reply With Quote
Old 12-12-2006, 10:52 AM   #7
Matthijs
Groupie
Matthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enough
 
Matthijs's Avatar
 
Posts: 199
Karma: 666
Join Date: Oct 2006
Location: Eindhoven, the Netherlands
Device: iLiad, DR1000S, DR800SG
Thanks for sharing. We _will_ port this back into ipdf mainstream, after some internal testing.
I will alert the software engineer working on ipdf to check this topic.
Matthijs is offline   Reply With Quote
Old 12-12-2006, 11:41 AM   #8
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
Quote:
Originally Posted by CommanderROR
I'm looking forward to seeing a news IPDF6 or something like it with a nice package done by Yokos...so that all the noobs like me can safely install it on 2.8...:-)
I'm investigating 2.8, they made some seriously incompatible changes in 2.8 that don't easily go back to 2.7 just by installing ipdf. They placed the scribble support into a shared library and appear to have updated all of the liber libraries in 2.8. So there's a fair number of files that have to be updated on 2.7 to install the 2.8 ipdf on it.

I have extensive customizations to my 2.7 but I'm thinking I'll have to upgrade to 2.8 so I can continue producing items of use to the mainstream. I'm also curious to see if they patched any of the kernel annoyances in 2.8.

Their lack of recognition of the legitimate need to serious debugging support is very tiresome. My kernel development would move much more rapidly if they'd get off their high horse. Their continued intransigence on releasing their grip on other people's software is getting old as well. Why do we have to install dillio? Why can't we just unlock minimo?

I'd like to release RTF support along with Graphic Novel support but iRex continues to be the bottle neck to those efforts as well.
scotty1024 is offline   Reply With Quote
Old 12-12-2006, 11:53 AM   #9
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
Quote:
Originally Posted by kapoira
thanks scotty i was pasing images to a pdf, and i was thinking that the background of the image wasn't pure white because the convert of the images to grayscale (i see that the borders of the iliad are more white), and i was thinking that the error was because to use a grayscale pdf in the ipdf of the iliad, now you have showed to me the problem was the ipdf no the pdf, thanks, thanks, and sorry for this next question,

you will put a new ipdf compiled with this problem resolved, thanks for the response if it yes i will wait if it's not i will try to compile it ;-)
I can post a 2.7 ipdf compatible with Yokos' excellent installer.

And as you point out and I forgot to mention, yes, with the new ipdf your white paper is indeed now white paper.

Paul's books look noticeably sharper with this new enhancement. I use his scan of Real Soldier's of Fortune as one of my standard tests.

We also got an able assist from George Bush on this effort as well.

I attach pictures to document the statements I made earlier in the thread.

Grey256 is a straight 256 level grey conversion.
Grey16 is a straight 16 level grey conversion.
Grey16fs is a 16 level grey conversion with FS error distribution.
Lena16fs is the "standard" image run through the same 16fs algorithm.
Attached Thumbnails
Click image for larger version

Name:	grey256.jpg
Views:	359
Size:	10.6 KB
ID:	2352   Click image for larger version

Name:	grey16.jpg
Views:	333
Size:	11.9 KB
ID:	2353   Click image for larger version

Name:	grey16fs.jpg
Views:	343
Size:	13.1 KB
ID:	2354   Click image for larger version

Name:	lena16fs.jpg
Views:	350
Size:	42.8 KB
ID:	2355  
scotty1024 is offline   Reply With Quote
Old 12-12-2006, 12:13 PM   #10
jęd
Evangelist
jęd has a complete set of Star Wars action figures.jęd has a complete set of Star Wars action figures.jęd has a complete set of Star Wars action figures.
 
Posts: 458
Karma: 293
Join Date: May 2006
Quote:
Originally Posted by scotty1024
Their continued intransigence on releasing their grip on other people's software is getting old as well. Why do we have to install dillio? Why can't we just unlock minimo?
I'm guessing thats more to do with developer time than any intransigence on Irex's part... Though nothing stopping someone porting Minimo...

Quote:
Originally Posted by scotty1024
I'd like to release RTF support along with Graphic Novel support but iRex continues to be the bottle neck to those efforts as well.
Whats the bottleneck...? The main one I can see is the association between filetypes being "hardcoded", but even that it seems could be got around.... Anyhow, Irex do seem a lot more communicative that earlier, which is always good...
jęd is offline   Reply With Quote
Old 12-12-2006, 12:34 PM   #11
henkvdg
Groupie
henkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of ithenkvdg has read War And Peace ... all of it
 
Posts: 180
Karma: 66830
Join Date: Oct 2006
Device: IREX iLiad, Pocketbook Pro 903
Lena16fs

Lena16fs on my iLiad looks more like a 3bit/pixel picture then a 4 bit/pixel picture.

I used microsoft Photoeditor to reduce the numer of tones to 8 and then it looks almost the same as on my iLiad. (with the image-viewer).
henkvdg is offline   Reply With Quote
Old 12-12-2006, 01:43 PM   #12
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
Quote:
Originally Posted by henkvdg
Lena16fs on my iLiad looks more like a 3bit/pixel picture then a 4 bit/pixel picture.

I used microsoft Photoeditor to reduce the numer of tones to 8 and then it looks almost the same as on my iLiad. (with the image-viewer).
The algorithm doesn't guarantee that all the available grey levels get used in a particular image.

It works in a manner similar to MP3 does for music, it pushes the visual error caused by the loss of grey levels into areas that the human eye doesn't pay as much attention to. Sort of like pushing dirt under the carpet to hide it.

Also, I deliberately used iRex's own chosen error distribution algorithm. I'm not saying it's the best (it isn't) but I figure it'll be quicker to get 2.8.1 of PDFViewer out of them by using it.
scotty1024 is offline   Reply With Quote
Old 12-12-2006, 01:48 PM   #13
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
Here's version 6 of ipdf using the new Mono 8 to Mono 4 error distribution technique.

I've also added a new feature. On startup it checks for the presence of the file /etc/speedkills. If that file exists you get the slower black flash between pages. If it doesn't, you get the faster creepy crawly matrixy effect.

I'm still refining my improved page transition code and need to now take into account the new 2.8 changes iRex have made.

I've also compiled this version with iRex's 2.7 toolchain rather than Antartica's to see if it's more stable for users.
Attached Files
File Type: gz ipdf_v6.tar.gz (21.4 KB, 273 views)
scotty1024 is offline   Reply With Quote
Old 12-12-2006, 09:59 PM   #14
scotty1024
Banned
scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.scotty1024 is no ebook tyro.
 
Posts: 1,300
Karma: 1479
Join Date: Jul 2006
Location: Peoples Republic of Washington
Device: Reader / iPhone / Librie / Kindle
Version 7.

Centered pages.
Working zoom.
Less useless debug messages so its faster.

Code for iRex...

Code:
void XMgr::drawImage(SplashBitmap * bmp, 
                     int xSrc , int ySrc , 
                     int xDest, int yDest) 
{
    // Get our source image width and calculate delta frem screen width
    int srcWidth = bmp->getWidth();
    int deltaX = SCREEN_WIDTH - srcWidth;
    if (deltaX < 0) {
      // Source too wide, trim to screen width
      srcWidth = SCREEN_WIDTH;
    }

    // Get our source image height and calculate delta frem screen height
    int srcHeight = bmp->getHeight();
    int deltaY = CLIENT_AREA - (srcHeight + yDest);
    if (deltaY < 0) {
      // Source too tall, trim to screen height
      srcHeight = CLIENT_AREA - yDest;
    }

    // Get src image pointer from SplashBitmap
    char *src = (char*)bmp->getDataPtr();

    // Handle partial bitmap (zoom)
    if (xDest < 0) {
      // Outdent left margin
      src -= xDest;
    }
    if (yDest < 0) {
      // Skip down image
      src -= yDest * bmp->getWidth();;
    }

    // Point to screen memory buffer
    char *dest = screenMem;

    // Fill page header with white
    if (yDest > 0) {
      memset(dest, 0xf0, yDest * SCREEN_WIDTH);
      dest += yDest * SCREEN_WIDTH;
    }

    // Fill left margin
    if (xDest > 0) {
      memset(dest, 0xf0, xDest);
      dest += xDest;
    }

    // Copy rows from image to screen
    for (int i = 0; i < srcHeight; i++) {
      // Copy source line of data
      memcpy(dest, src, srcWidth);
      src += srcWidth;
      dest += srcWidth;

      // Fill right/left margin with white
      if (deltaX > 0) {
        memset(dest, 0xf0, deltaX);
        dest += deltaX;
      } else {
        // Skip extra source line material
        src -= deltaX;
      }
    }

    // Fill bottom of page with white
    if (deltaY > 0) {
      memset(dest, 0xf0, deltaY * SCREEN_WIDTH);
    }
}
Attached Files
File Type: gz ipdf_v7.tar.gz (18.7 KB, 276 views)
scotty1024 is offline   Reply With Quote
Old 12-13-2006, 02:42 AM   #15
Matthijs
Groupie
Matthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enoughMatthijs will become famous soon enough
 
Matthijs's Avatar
 
Posts: 199
Karma: 666
Join Date: Oct 2006
Location: Eindhoven, the Netherlands
Device: iLiad, DR1000S, DR800SG
@scotty: it would be beneficial to all of us if you supplied the sources to your versions of ipdf. I've branched the 2.7 version in our public repo so your changes can be easily made visible and ported.

Quote:
Originally Posted by jd
Anyhow, Irex do seem a lot more communicative that earlier, which is always good...
Do you see any relation between the absence of the backstabbing, provocative and "look at me being superior and them being stupid" remarks on this forum and my drive to spend my precious time helping here?
Matthijs is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Fixing Up Typography ahi Workshop 65 11-18-2013 04:35 AM
DR1000 Fixing the reset button? Uthred iRex 5 09-14-2012 11:11 AM
Sony is fixing the Available Soon issue JSWolf Sony Reader 20 01-07-2010 12:28 PM
Fixing paragraphs with calibre? enarchay Calibre 17 08-16-2009 08:31 PM
iLiad iRex Fullscreen mode iPDF realityloop iRex Developer's Corner 4 07-12-2007 08:52 AM


All times are GMT -4. The time now is 01:52 PM.


MobileRead.com is a privately owned, operated and funded community.