Shiny New E-Book Gizmo: The Amazon Kindle


View Full Version : Introducing the new Sony-Reader-speed ipdf


scotty1024
10-26-2006, 04:21 PM
How would you like your iLiad if it could flip pages as fast as the new Sony Reader?

You can have it today if you're still on 2.7. I'll email iRex tonight and see how fast they can push out a new ipdf for everyone else.

Yeah I know, I was supposed to develop a new idjvu, but the bottle neck called to the X developer in me...

Brick factor: zero. Hey it's iRex source using iRex's tool chain!

To install:
root@ereader:/# cd /usr/bin
root@ereader:/usr/bin# mv xpdf xpdf.orig
root@ereader:/usr/bin# cp /mnt/card/ipdf xpdf

Enjoy the speed, I suggest goggles...

And yes, this should be a wake up call for Sony and their attitudes towards their user community as well.

NatCh
10-26-2006, 04:27 PM
And yes, this should be a wake up call for Sony and their attitudes towards their user community as well.
Are you referring to the fact that iRex finally released (most of) their Source?

Or are you calling for Sony to block access as quickly as iRex has been doing? :tongue3:

scotty1024
10-26-2006, 04:34 PM
They released more source?

Anyways, you may find the text to be a bit sharper now too. :D

tribble
10-26-2006, 04:46 PM
What did you change?

/* cries for having updated to 2.7.1 before it was safe */

scotty1024
10-26-2006, 04:58 PM
Cheer up, if you're on the beta you may see it real soon. :)

I'm shipping the source off to iRex for their consideration. I'll post it tonight in Alex's new upload area.

vranghel
10-26-2006, 05:04 PM
Wow! Congratulations Scotty! The text sharpness is great! I really do hope that iRex will implement your code ASAP.

scotty1024
10-26-2006, 05:08 PM
OK rev 2 available now.

This version reduces startup time and reduces memory foot print.

NatCh
10-26-2006, 05:15 PM
And to think where we (okay, y'all) would be if the ... folks at iRex had just decided to play well with others back in June....

For your next trick, scotty1024, howsabout you solve that little power management thing they've had so much trouble with? :mad:

They ought to pay you. No really, they should. :yes:

Aenea
10-26-2006, 05:25 PM
How would you like your iLiad if it could flip pages as fast as the new Sony Reader?

Ehrm, thanks for making this!! Since I'm a programmer myself and since I had a look around in ipdf this evening I wonder what you changed...

Also, I simply installed it and opened a pdf, the text was skewed. The left-margin starts at the top-left corner and end at the bottom-right. Probably a small bug...

Aenea

CommanderROR
10-26-2006, 05:35 PM
Scotty...you are amazing!!!

I can't try this because I'm on 2.7.1, but since I'm a beta-tester...maybr I'll get to see it soon. Really great stuff!!!

I think we should really conisder starting some kind of paypal donation thing for our developers here on the Forums. I doubt iRex is going to be paying you and others, so maybe that would be an option?

btw....can somebody post a short vid of this beast in action?

Maybe tackling that power-issue would be a worthy goal for you scotty...:-)

scotty1024
10-26-2006, 05:39 PM
@Natch

Well faster rendering and faster drawing (I made both faster) means less CPU cycles per page. Fewer CPU cycles per page generally means more electrons left in the Lithium Ion battery for later. So I've helped things a bit with this new code.

Mike Kostousov
10-26-2006, 07:32 PM
Hmm... Something is going wrong. I think - bytes_per_line in XImage rendering a bit less, that we realy need. I get such shifting on several pdfs... Including iRex release notes :)

Aenea
10-26-2006, 07:38 PM
Hmm... Something is going wrong. I think - bytes_per_line in XImage rendering a bit less, that we realy need. I get such shifting on several pdfs... Including iRex release notes :)

Yup, same problem I had.... (darn it, must buy new digi camera :))

scotty1024
10-26-2006, 07:54 PM
I saw it on the way home, its only with scanned PDF documents. And I've got an idea of what it is after playing with about 30 documents.

Let me finish dinner and I'll push out rev 3.

The bottle neck was very simple. iRex had setup Splash to render 24 bits per pixel and then they were using a cubic color space reduction on the rendered raster to convert it for X.

I changed the code to render 8 bits per pixel static grey scale (which speeds up rendering) and just hand that bitmap off to X which can easily convert it to 16 level grey scale (speeding up drawing by removing the cubic color space reduction.) I was also able to reduce foot print and speed startup by removing all the color space stuff the cubic reduction required.

I believe the current issue is a padding problem that I can easily solve. And reminds me I need to test more than just my own text based PDF's. :D

By the way, has anyone noticed how beautiful the PDF's are now? I mean the text ones of course. :)

Try one of the documents from manybooks. Stunning, I can't believe I didn't notice how hazy they were before. That was the cubic color space reduction spreading the color error around mushing up the edges as it did that.

I also tried a couple magazines (which are scanned but do work, giving me the clue as to my problem) and they are noticeably sharper as well.

scotty1024
10-26-2006, 07:57 PM
Maybe tackling that power-issue would be a worthy goal for you scotty...:-)

All I can say at this point is that I got an email this morning. Too early to say yet but it seems promising...

scotty1024
10-26-2006, 07:58 PM
Hmm... Something is going wrong. I think - bytes_per_line in XImage rendering a bit less, that we realy need. I get such shifting on several pdfs... Including iRex release notes :)

One of the things I inherited was a setting to Splash that I think is padding the lines to a 4 byte boundary. I'm going to set it to 1 and see what happens, after I have dinner.

Mike Kostousov
10-26-2006, 08:15 PM
When I was playing with xpdf, I've found that non-24bits splash doesn't work properly with several pdf. For example, 32bits splash (it could be very useful for construct QImage, directly from Splash) didn't render color background well. I don't know why. But, may be I was wrong, may be xpdf was wrong and now it is fixed in poppler. May be iRex guys didn't look deep inside... :)

scotty1024
10-26-2006, 08:34 PM
It was exactly what I thought it was.

Padding is now set to 1 and all is happiness.

My next performance tweak is my Famous Jack Rabbit Performance Test.

scotty1024
10-26-2006, 08:39 PM
When I was playing with xpdf, I've found that non-24bits splash doesn't work properly with several pdf. For example, 32bits splash (it could be very useful for construct QImage, directly from Splash) didn't render color background well. I don't know why. But, may be I was wrong, may be xpdf was wrong and now it is fixed in poppler. May be iRex guys didn't look deep inside... :)

If you have one of those PDF's lying around please test it on the newly enhanced ipdf and let me know how it works?

I'm off to put in an hour on the promised idjvu viewer.

DHer
10-26-2006, 11:43 PM
Scotty, you are doing magic.

I can't wait for this day to end, so i can spend some quality time with my iliad and your releases.

scotty1024
10-27-2006, 12:39 AM
Scotty, you are doing magic.

I can't wait for this day to end, so i can spend some quality time with my iliad and your releases.

Thank you and enjoy the code when you get home! My wife has been accusing me of giggling as I sit there flipping pages on my iLiad. Its really just "fast enough" now.

I just sent an electronic package off to iRex and I'm heading for bed myself.

tribble
10-27-2006, 02:06 AM
scotty, i hope iRex will use your code really quick. Cant wait to see it myself. :D
Thanks for your hard work.

Matthijs
10-27-2006, 02:48 AM
This looks very interesting, however:

The reason why we stuck to RGB rendering (yes, we did try this Mono8 too) is that now images inside the PDF document that were in color will look ugly (they show 'banding', take a look at the first page of the iLiad User Manual for an example).

If this dithering inside libpoppler can be fixed, we can have another look at this approach.

It was also reported by one of our engineers that some of the smaller fonts actually look LESS sharp in 8bit grey. But that is of course hard to compare with only 1 iLiad...

As for the speed: going from page to page uses pre-rendering and is not affected by the changes above. Only the actual rendering of pages uses more memory and is slower in 'our' implementation, but pushing the resulting XImage to the screen (which is all that is done when a page is flipped) is equally fast. Startup time of the viewer and rendering of a page that is not pre-rendered is faster of course.

In summary: there are outstanding problems to be fixed. Dithering of color-images in Mono8 mode in libpoppler needs adjustment for the iLiad color-space, that is a very important one.

We really hope that by working together with the community on these kind of things, we can bring the iLiad software to the next level even faster.

Thank you all for contributing, please continue to do so.

Matthijs van de Water
Embedded Software Engineer
iRex Technologies BV

emkay
10-27-2006, 03:25 AM
Matthijs,
I think your attitude is really great. This collaboration with the developer community is really going to pay off for everybody, iRex and iLiad users.
Well done iRex! Well done Scotty and other developers! :crowngrin

Matthijs
10-27-2006, 03:33 AM
I've just had another look at the code:

What we need to add (inside ipdf) is the Floyd-Steinberg dithering that was there before, but now on the 8bit greyscale image. The implementation of that algorithm in greyscale should be easier than the one that was used to convert from RGB24 to 8bit greyscale including dithering.

You should understand that the iLiad does a linear conversion from 256 grey levels to 16 grey levels on a device-driver level (so the first 16 levels of grey become white, etc.).

Maybe Scotty can have a look at this when he wakes up ;)

Mambo
10-27-2006, 05:08 AM
Matthijs:

I think the majority of users would be more happy with faster rendering than with nice looking colour PDF images. Anyway we have only 16 colours.

I have PDF documents that take more than 16 seconds to render sometimes. Even if I read in sequential order, sometimes I want to skip pages and flip them over, when prerendering just fails.

Actually in the iLiad settings there should be a switch: faster PDF viewing or better looking images. The user can choose which one he wants.

please give the user choices! It will not make this device overly difficult to use! It is up to you whether you go into the settings or not, and also you could any time go back to default as an option, this way there would be nothing to worry about.

I can not see why this would be a problem.

CommanderROR
10-27-2006, 05:39 AM
I think the idea with the "options" would be nice, but if you take a long-term view, it will get confusing because every user has his/her own ideas and you would probably end up including more options than any single and sane person could ever want...:-)

I do however believe that we should get a choice of using Scotty's new PDF viewer if we want to. For me it wouldn't be a problem if there were some display errors in the pictures, because I use text-PDF files only, the maximum in image-display being a frontcover and maybe a map in most cases. I also founf that the curretn implementation doesn't really convert colours to grey all that well anyway so I started converting my PDf files to greyscale before I transfer them to the Iliad, thus also reducing file-size...

Matthijs
10-27-2006, 05:40 AM
Basically making a setting configurable through the settings application takes up valuable development resources. It sounds simple, but it actually takes quite some work (GUI, libraries, testing).

What I'm suggesting is taking a little time to implement the Floyd-Steinberg algorithm for 16 levels of grey and have 1 single solution which is both faster and has good-looking images.

jęd
10-27-2006, 05:43 AM
please give the user choices! It will not make this device overly difficult to use! It is up to you whether you go into the settings or not, and also you could any time go back to default as an option, this way there would be nothing to worry about.


Perhaps if there are two viewers for a format a "Open with...?" dialog would be handy. That way the user can choose whether he wants fast pages or nice pictures... :D

emkay
10-27-2006, 06:11 AM
I think Matthjis is suggesting that with tweaking we could have fast pages which also look nice. I suggest we focus everybody's effort on making one great PDF viewer, instead of having various viewers with different strengths...
(For the moment anyway. I know there should be more software options further down the line.)

kusmi
10-27-2006, 06:45 AM
I think so too: Making things configurable should be the *last* resort, not the first one... if iRex can make pdfs rendering fast and nice, there is no need to have an user-configurable option...

just my 2 cents...

arivero
10-27-2006, 07:42 AM
You should understand that the iLiad does a linear conversion from 256 grey levels to 16 grey levels on a device-driver level (so the first 16 levels of grey become white, etc.).

Ah, that was the improvement letting iRex to use the X server with the framebuffer. Xserver crashs heavily when you attempt to play with (theoretically implemented) less than 8-bits levels. In the librie the only working xserver was the xvfb, the virtual framebuffer, and I was forced to take screenshoots of the virtual one and transfer to the real.

scotty1024
10-27-2006, 08:25 AM
The technical term for what is occurring is posterization you can read more about it here: http://www.cambridgeincolour.com/tutorials/posterization.htm .

The various error distribution schemes (Floyd, bi-cubic) work in a manner similar to MP3, except for the eyes instead of the ears. They attempt to hide the loss of data through perceptual tricks. The various methods work better with some documents than with others and their effectiveness various from one person to the next.

One thing though is generally accepted: using any of them on text glyphs is not a "good thing".

As others here had already commented upon elsewhere here, text in PDF's was at times not looking too sharp on the iLiad. When I removed the error distribution I immediately noticed the extra sharpness. After one struggles as hard as I did to get a working unit one tends to try and think more fondly of his sickly device eh what? :(

Rather than applying error distribution, which for black text there are no errors needing to be distributed (but we find something to distribute anyway :)) these methods should be applied where they make the most sense: on images.

Everyone needs to keep in mind what happened yesterday. I took a cold toolchain, got it working, took a cold tar ball and got it to compile. I then walked into that cold source and found a big flashing sign that read "big bottle neck right here". The source didn't say "This code is sub-optimal because we found trying to render to 8 bit grey scale resulted in the following ...."

I fixed the bottle neck, then popped out two more revisions.

Yes I noticed some things were better, in particular some scanned magazines I have looked much better. But other scanned items looked worse. I also noticed that these issues seemed to track with the compression method being employed.

I marked this one down for "further investigation", because as others have already commented: when you go from 16.7 million colors down to 16 (1 million to 1), some things just aren't going to look right. :D

I've had an iLiad benchmark I've been calling the Jack Rabbit Benchmark. Its a PDF of a menu from a local catering outfit. Performance of this simple two page document on the iLiad can kindly be described as "sub-optimal". In the process of fixing this document's performance issues I believe I will turn up the source of some of the image issues.

I think I'm going to find that poppler has some "growth opportunities" in its code when it comes to rendering images into 8bpp grey.

Can I create a faster error distribution routine for the iLiad? You bet, I'll get out my PXA-255 manual and throw some wireless MMX opcodes into the code to speed it up nicely. But I only want to apply that to images, not glyphs.

In the mean time no one is paying me to work on this stuff: I owe, I owe, its off to work I go... :)

CommanderROR
10-27-2006, 08:55 AM
So you do have a job as well...I was wondering...:-)

If you can create an "adaptive" routine that applies the correct "downsampling" for images and leaves text alone then all the better.
I was wondering about the bad text-sharpness ever since I read the Iliad in really bright (sun-)light for the first time wher eis was very visible. I thought it was due to some suboptimal anti-alias thing...

I really hope you and/or iRex will find a way to improve sharpness, page-turn speed and retain image quality (for those who need it) all in one package...and I wish you all the luck you're going to need.

At the moment I really envy you coders out there...but on the other hand...it#s nice to sit back and watch others work... :vulcan:

Matthijs
10-27-2006, 09:59 AM
I'm sure that with some collaboration, we will be able to fix the rendering of pictures in the poppler library, after which the switch to use 8bit grey internally is a logical one.

I merely wanted to point out to the others that it might be a little longer before this turns up in a release version, we need to fix the issue with images first.

That being said, I will send Scotty some more info via email. Just in case he gets bored this weekend ;)

Mambo
10-27-2006, 11:45 AM
I think so too: Making things configurable should be the *last* resort, not the first one... if iRex can make pdfs rendering fast and nice, there is no need to have an user-configurable option...

just my 2 cents...

Sure,

But consider that I have a lot of scanned pdf files - you could have too - and I don't need a fancy algorythm to do the deposterisation, because those are text only (even if scanned in colour, etc.)So the computer will be able to differentiate between scanned text and real images? No way, that is my two cents...

So please Irex, rather give user-configurable option, and I don't think it should be the last step in development otherwise it will never be ready, because a software is never ready.
And if you can not add economically features to the existing settings application then I suppose a new settings application could be easily added or something
(or what about an icon saying: draft mode?)
I am also developing software and never found difficult to add 1 or 2 more tickboxes or something.

Otherwise there are less user-configurable options on the iLiad than on an average mobile phone, do you think this is right? Everybody truly thinks that the system should be developed for dummies only? (sorry for the word) And what about the rest? shall we suffer just because others are not able to reset their devices and go back to default settings? Is it democratic to give no options to the user at all? Is it politically correct?
I don't understand the philosophy behind it, if you give an option somewhere in the settings it will help some people, while it does not take ANYTHING away from other people so why do they complain?

Why should you optimalize an algorythm to work faster, when in most of the time you don't actually need the deposterizer algorythm, so you could just switch it off. There will be always situations when you can not eliminate the trade off between nice rendering and fast rendering.
Also you could save valuable devlopment time for something else (Power management?) and/or produce an earlier update IMHO.

k2r
10-27-2006, 01:04 PM
And if you can not add economically features to the existing settings application

You don't really want a system wide setting for this but this feature belongs into the PDF-Reader (and the manifest.xml) as do other viewing options (scaling, panning).

And yes - since I depend on books that are images and books that are text, too, I'd be very pleased if we could get this option, soon. It would be much of an improvement.

Now, if besides of compiling the software of my choice I could even use it...

scotty1024
10-27-2006, 02:07 PM
I agree, some of these options could go into manifest.xml and the toe tag could be updated to edit those new options.

We could then add several error dispersion routines to choose from for those documents where you need it to look its 16 grey's best.

scotty1024
10-27-2006, 02:22 PM
I'm sure that with some collaboration, we will be able to fix the rendering of pictures in the poppler library, after which the switch to use 8bit grey internally is a logical one.

...

That being said, I will send Scotty some more info via email. Just in case he gets bored this weekend

iRex has pointed out several times: its a book reader, not a PDA.

In this case I'm trying to point out: its 16 grey, not 16.7 million color.

The number one priority on a 16 greys eBook reader is glyphs. The text must be impressively sharp and crisp.

If someone comments negatively on a picture you just look at them and say "Hey, 16 greys, they'll have the color one out next year." Heads will nod sagely and go back to admiring the crisp glyphs.

I haven't gotten your email yet, but I found an X drawing speed up to make that I suspect you'll agree is worthy. You can even drop it into the 24bpp ipdf.

Hopefully that'll keep you from being bored this weekend as well. :)

kusmi
10-27-2006, 02:30 PM
@Mambo: Look, there are different kind of users (as well as developers :-) ) And I agree, at the moment there is almost *no* configuration on the iLiad, which is also sub-optimal, there should be a balance. Without starting this typical windows/mac/linux comparison thingy, but there you see all the different conclusions about end-user configurability - and of course taste is different - but personally (not generally, this is my personal view on things, as software developer and graphical designer) I don't want a device, with 100 of config-options which pollute the config-screen, if there is a way around this, doing some decent development work.

But of course, there could be some "pro"-settings in the manifest.xml, so advanced users could tune the content, this is fine with me - I would just not bother the standard user with rendering options on the iLiad itself...

DHer
10-28-2006, 02:20 AM
I finally had a chance to try it. The fonts really look better.

Concerning speed, i can't describe the difference.

Anyway, thank you, scotty. I'm also really looking forward to the djvu viewer.
Maybe i'll fire up the toolchain after breakfast.

deadite66
10-28-2006, 02:41 AM
can anyone make a vid of the speed difference?

<- 2.7.1 :<

k2r
10-28-2006, 04:41 AM
iRex has pointed out several times: its a book reader, not a PDA.

The hardware is clearly capable of being a PDA. And if the SDK comes out finally it's up to us to define "what" our personal iLIad is
Of course it's not up to iRex to develop applications it does not see a market for.
That's the beauty of free software.

Personally I'd really like to have a PIM application on the device.
And an IMAP mailreader.
And better HWR.
One of the interesting parts would be developing a good GUI for all of this.

#!chris

R2D2
10-28-2006, 07:30 AM
Excuse me guys and let me give you some out of the box input:

There is a company called Picsel (http://www.picsel.com/) which produces viewers for graphics, HTML, XLS, DOC and PDF for embedded devices. The software, called Picsel Browser, is also available for the Sharp clamshell VGA Zaurus which has hardware very similar to the iLiad:

http://gallery.mobileread.com/data/509/medium/arm_devices.jpg

The software is extremely fast and the display of the files is of a very high quality.
Pan and zoom is no problem at all, regardless of the document format:

http://www.drdroth.de/picselbrowser.jpg (http://www.drdroth.de/picselbrowser_divx.avi)
click on image to download video...

So why don't you call them and ask them to make a iLiad version of the Picsel Browser (essentialy a simplified, low frame rate version of the existing Zaurus one)? This would have some advantages:

* Rapid and rather inexpensive development, because a Zaurus version does already exist.
* No need for Minimo anymore, because HTML support is already supported by the Picsel Browser. Web-Browsing is also possible.
* No need to create a pan and zoom solution on your own.
* Support for office documents. Not that unimportant in my opinion...
* Extremely high performance rendering.

This approach would be much less dirty than trying to implement all kinds of improvements into the existing open source viewers.

In case that you have never seen the Picsel Browser on Zaurus, you have to see it with your own eyes.
So click onto the above screenshot to download a divx movie I created some time ago.

Sorry for repeating myself. Opinions?

btw.:

Scotty you are either drinking to much coffee or having too much spare time and/or a very understanding wife/girlfriend.
In any way, please continue to do so. ;-) Great, impressive work!!!

CommanderROR
10-28-2006, 10:14 AM
Is the Picsel Software Open Source or is it it "Payware"?

If it's normal, commerical stuff, then iRex would probably have to get some kind of agreement going with them if they want to include that Software into the Iliad...
Since there is no way for a user to install Software (apart from the hackers way...^^) on the Iliad apart from the IDS, I'm not sure how easy it would be to get something like this growing...and I don't know how much interest the Picsel guys would have in supporting the Iliad either...

R2D2
10-28-2006, 10:18 AM
Irex would have to pay for it. But in my opinion it would be worth it, because it is a very advanced piece of software which solves some of the current problems and which would free some development resources for tasks, which are more important than imperfectly reinventing the wheel (that's a gross exageration, I know, but it helps to understand my point).

So let's invest money into an iLiad Picsel Browser and concentrate on other things.

CommanderROR
10-28-2006, 10:21 AM
It might be worth a try. Will you post it at the iRex Forums? I think it would be better to have it there than in Scotty's PDF viewer thread...:-)

R2D2
10-28-2006, 10:42 AM
Ok, put it on the IREX forum. But I am sure that even with Picsel Browser installed there would be enough things left for Scotty to be improved. ;-)

vranghel
10-28-2006, 12:34 PM
Wow! That browser is truly impressing. I really wish that something like that will be implemented into our iLiad. It would make reading so much more enjoyable/easier.

Now the question is if irex would be willing to purchase licenses (or some sort of agreement) for this browser. That means some extra money coming from who? :huh:
Probably those of us who want to use it.

R2D2
10-28-2006, 12:47 PM
Well, development does cost money as well. It would be a radical approach, but it would probably make sense, even economically.

scotty1024
10-28-2006, 02:36 PM
Motorola bundles Picsel on the Motorola Ming (a1200) cellphone (my wife and I both have one.)

Motorola opts to use Opera as their web browser on the phone, 'nuff said there. :)

I'e found the tool useful for pulling up a document for reference purposes but using it as a reader has always been too tedious.

It would also require some non-trivial customizing to fit into iRex's plumbing, all of which I'm sure Picsel would be more than happy to do per hour for $$$. And of course anytime iRex wanted to change anything, I'm sure Picsel would be more than happy to do it per hour for $$$. And if Picsel had any bugs in their code I'm sure they'd be happy to fix those per hour for $$$ as well. :)

R2D2
10-28-2006, 04:00 PM
Motorola bundles Picsel on the Motorola Ming (a1200) cellphone (my wife and I both have one.)

Motorola opts to use Opera as their web browser on the phone, 'nuff said there. :)

It works perfectly well to view HTML pages. Of course Opera offers more options for real web-browsing. But to view HTML I prefer the Picsel Browser.

Of course using it on the iLiad will cost money. You can probably ask Sharp how much it does.

On the Zarus people use QPDF, a very nice programm. But people did nevertheless send heaps of money (135 Euro) to Japan to get the Picsel Browser. Nuff said?

Rendering speed alone would be enough reason for me to make a call to Picsel Technologies. Take a look at the video.

Of course an alternative would be to find out what is going on behind the scenes of the Picsel Browser and then try to rewrite XPDF and all the other viewers. IMHO the other way around of supplying reader for DRM and so on for a Picsel Browser seems to be much more promising for me. Because at the end of the day I need a perfect software for a perfect hardware and not just a perfect hardware.

scotty1024
10-28-2006, 06:21 PM
Rendering speed alone would be enough reason for me to make a call to Picsel Technologies. Take a look at the video.

Of course an alternative would be to find out what is going on behind the scenes of the Picsel Browser and then try to rewrite XPDF and all the other viewers. IMHO the other way around of supplying reader for DRM and so on for a Picsel Browser seems to be much more promising for me. Because at the end of the day I need a perfect software for a perfect hardware and not just a perfect hardware.

Technically the iLiad isn't using xpdf, its using poppler with an iRex supplied front end. I've got a couple more speed ups for the iRex code that affect actual drawing to the screen time that will shave a bit off the throwing up the raster time.

But at this point, after converting to 8 bit grey with no ED, we're rapidly reaching the point of diminishing returns vs the e-Ink panel update overhead. They really did do a good job on their threaded rendering engine (other than not planning for re-use). Running 8 bit lean like it is now I have a hard time out pacing it.

scotty1024
10-30-2006, 05:00 PM
OK, here is the rev 4 of ipdf.

The major enhancement here is using shared memory to pass the image from ipdf to the X server instead of shoving it down the Xlib pipe to the X server.

I also removed the fill rectangle code and handled its function in the code that transfers the Splash bitmap to the shared memory buffer. The shared memory buffer is a full sized CLIENT_AREA buffer. If the Splash bitmap is too narrow and/or too short I fill the unused areas in the buffer with white bytes (rather than fill the whole client area with white, then paint in the bitmap.)

Just that extra bit quicker now, you can see it, but you can't describe it, but you know you want it. :)

Enjoy.

tribble
10-30-2006, 05:09 PM
Scotty, you are making me cry :(

I want to see it. Can you make a short vid?

Oh, and are ou planning to try and implement that dithering, that matthijs mentioned? Then we might all get to enjoy your improvements with the next update.

scotty1024
10-30-2006, 05:55 PM
iRex and I have been having a very cordial email exchange back and forth on several topics.

They've explained why images must look good in PDF now and I fully support their business needs in that department. We're discussing how I could best assist them in modifying Poppler.

In the mean time I'm polishing what I have as best as I can.

Sources will be in the project's area here on MR later tonight.

scotty1024
10-31-2006, 03:47 PM
In case anyone was wondering "Is shared memory really a good thing for sending bitmaps to the X server?"

I thought I'd share this comparative pair of traces.

This is the new ipdf rev 4 sending the bitmap to X. Writes 44 bytes, gets two 32 byte responses.

write(22, "\201\3\n\0\26\0\340\0%\0\340\0\0\3\247\3\0\0\0\0\0 \3\247"..., 44) = 44
read(22, "\203u\226\0z\17>\2408\0\0\0\25\0\340\0\0\0\0\0\1\0\1\0"..., 32) = 32
read(22, "\1\1\230\0\0\0\0\0\25\0\340\0\234\325\1\0t:\r\0h:\ r\0\274"..., 32) = 32

This is the 2.7 ipdf sending the bitmap to X. Those resource unavailable's are the X server telling the client "back off man the fire hose is choking me!" :D

You'll also notice it starts by trying to write 261888 bytes but X only accepts 65280 bytes due to a limit in the socket buffer size.

writev(22, [{"8\1\4\0 \0\340\0\4\0\0\0\377\377\377\0F\0\5\0\26\0\340"..., 60}, {"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 261888}], 2) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 196668}], 1) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 131388}], 1) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 66108}], 1) = 65280
writev(22, [{"\0\0\0\0\0?\277\377\377\377\377\377\377\377\377\37 7\377"..., 828}], 1) = 828
writev(22, [{"H\2\306\377\26\0\340\0 \0\340\0\0\3U\1\0\0X\1\0\10\340"..., 24}, {"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 261888}], 2) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 196632}], 1) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 131352}], 1) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=22, events=POLLIN|POLLOUT, revents=POLLIN}], 1, -1) = 1
ioctl(22, FIONREAD, [32]) = 0
read(22, "\203u\n\1i\230G\2408\0\0\0\25\0\340\0\0\0\0\0\1\0\ 1\0\1"..., 32) = 32
poll([{fd=22, events=POLLIN|POLLOUT, revents=POLLOUT}], 1, -1) = 1
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 131352}], 1) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 66072}], 1) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=22, events=POLLIN|POLLOUT, revents=POLLOUT}], 1, -1) = 1
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 66072}], 1) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 792}], 1) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=22, events=POLLIN|POLLOUT, revents=POLLOUT}], 1, -1) = 1
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 792}], 1) = 792
writev(22, [{"H\2F\271\26\0\340\0 \0\340\0\0\3\367\0\0\0\255\2\0\10\340"..., 24}, {"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 189696}], 2) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 124440}], 1) = 65280
writev(22, [{"\377\377\377\377\377\377\377\377\377\377\377\377\3 77\377"..., 59160}], 1) = 59160
write(22, "8\2\4\0 \0\340\0\4\0\0\0\0\0\0\0C\0\5\0\26\0\340\0 \0\340"..., 40) = 40
read(22, 0xbffff77c, 32) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=22, events=POLLIN, revents=POLLIN}], 1, -1) = 1
read(22, "\1\1\22\1\0\0\0\0\25\0\340\0@\353\1\0t:\r\0h:\r\0\ 2749"..., 32) = 32

That's about as night and day as "This is your brain, this is your brain on drugs." :D

NatCh
10-31-2006, 04:06 PM
This is your iLiad, and this is your iLiad on downers?

scotty1024
10-31-2006, 05:24 PM
I want to see it. Can you make a short vid?

Well no vid but I did figure out how to do screen captures and posted some over in the idjvu thread (I spaced making a new thread, sigh).

arivero
11-01-2006, 11:41 AM
I was looking at the sources of ipdf, mainly to see where did they patch the /Launch hole. Man they have needed to reorder everything to translate xpdf to ipdf !? Or is there a g++ version of xpdf already around?

I now approve the idea of building idjvu over ipdf, if only to use the effort of going object oriented.

Antartica
11-02-2006, 09:19 AM
I was looking at the sources of ipdf, mainly to see where did they patch the /Launch hole. Man they have needed to reorder everything to translate xpdf to ipdf !? Or is there a g++ version of xpdf already around?

xpdf 3.00 is a C++ application (and somewhat heavy in the C++ aspect, at least at casual inspection). What you say is that ipdf is further abstracted?

arivero
11-02-2006, 10:00 AM
xpdf 3.00 is a C++ application (and somewhat heavy in the C++ aspect, at least at casual inspection). What you say is that ipdf is further abstracted?

Yep. I asked at irex, it seems they did the reshaping due to memory requeriments.