Sony Reader for PC -- not rendering 16-color png

[GMcG]

@grannyGrumpy
When I did that 256-16-256 colors workflow, I used a greyscale drawing of my own with about 230 colors.
Now I had a look at your Huck Finn drawing. It has only 37 colors! I think the others are of the same kind. So why do you want a 256 color png or a 100% quality jpg for this? I did some testing with Huck Finn and all I see is some blurring when zooming in the picture to double size and I don't know if this is a result of the quality of the picture or of the zooming. And some grey pixels are missing in the picture with 30% quality, but you don't see it in the original size of the picture. If you need to reduce the file size, I would make it with a compessed jpg with 50% quality.

George

[Jellby]

Originally Posted by GMcG

I did some testing with Huck Finn and all I see is some blurring when zooming in the picture to double size

Well, I definitely see JPEG artifacts in the shadowed parts of the hat, at both compression levels. Maybe you need to increase your monitor's brightness to see them?

By the way, for images with a reduced number of colors, PNG could probably be more efficient than JPG. At least, PNG could give a sensible filesize without any compression artifact.

[GrannyGrump]

But reduced-color png brings us back to square one ---- the ADE engine in some devices can not properly render 2-bit (2-colors) or 4-bit (16 colors).

So it looks like being stuck with either 8-bit 256-color png with its larger file-size, or with crushed jpegs in 256-greyscale, smaller files but compression artifacts.

But the 256->16->256 png makes a pretty small file, so probably would be the way to go with images with low color count.

[Tex2002ans]

grannyGrumpy, I just took a look at your test EPUB in your first post... Perhaps it might be best if you stick a few different types (4 gray, 16 gray, 256 gray) of the same exact image in the EPUB. This would make it easier to A/B/C compare on a device.

I specifically remember running into a serious "banding" problem on my Nook with some high resolution formulas which I reduced to 4-color grayscale (I assume 2-bit PNG?)

I will have to hunt down that EPUB and get a lot more details.

(Hopefully) tomorrow, I can get post more thorough testing/posting of my grayscale PNG samples.

[PeterT]

One thing that people might find interesting is this eBook in the Library here --> https://www.mobileread.com/forums/sho...d.php?t=211123

It's been used by people in the Kobo forums to look at items like font support / image rendering with different Kobo firmwares.

** EDIT

Bother; that link was not to the Font Tester I remember, and I'm currently not able to find the original post. I've attached the one I was thinking of that has images as well.

[Jellby]

Originally Posted by grannyGrumpy

But reduced-color png brings us back to square one ---- the ADE engine in some devices can not properly render 2-bit (2-colors) or 4-bit (16 colors).

Not necessarily, one thing is the number of colors, another thing is the number of bits used to store each color. You could have a plain grey image, that's just 1 color, but it cannot be stored as a 1-bit image (that's 2 colors, 2-bit would be 4 colors), because with 1 bit the colors are black and white (that is, unless we use an indexed mode, with a 2-color pallete, but 8 bits per color)...

But the 256->16->256 png makes a pretty small file, so probably would be the way to go with images with low color count.

That's basically what I mean. With 256->16 you reduce the number of colors to 16, with 16->256 you ensure that the colors are stored as 8-bit numbers.

[GrannyGrump]

@Jellby --- upon reading your post again, I realized that I didn't quite get your meaning the first time through. I didn't mean to sound argumentative, sorry.

@Tex2002ans --- Just for kicks and grins, here is another test-bed epub with identical images. They are 500px wide x 740 high (30% of the source image).
It includes 256-Greyscale (8-bit), 256 indexed palette (8-bit), 16 indexed (4-bit), 8 indexed (3-bit), 4 indexed (2-bit), and 2-color black & white (1-bit), AND, 16-color increased back to 256. (I threw in a couple of jpegs too, just for your viewing pleasure )

Once again, ADE for desktop shows everything just fine, but SonyReader for desktop barfs on anything less than 8-bit. ---- Well, not quite, the Sony software display of the 2-color black-and-white image is good. Go figure...

[Tex2002ans]

I have attached three EPUBs to this post:

• A modified GrumpyGranny with a few more derivatives I generated.
  • I generated 3 more sets of each image: Yours -> GIMP (Double checked Index values and saved), GIMP -> ScriptPNG, Yours -> ScriptPNG.
  • I added the bit depth to text above each section.
• A test EPUB with lots of variations of formula images that I generated.
  • 5 formulas * 6 variations = 30 images. Plus at the end I added ScriptPNG versions of ALL 30 images.
• An EPUB I created when I first ran into this problem (QJAE).
  • This was when I first ran into this "banding" issue. [Images at the end of the post]

Let me start off with a basic overview of the different bit Grayscale/Indexed PNG:

If you are saving as a Grayscale PNG, there is this:

1-bit: 2 shades of gray
2-bit: 4 shades of gray
4-bit: 16 shades of gray
8-bit: 256 shades of gray
16-bit*: 65535 shades of gray

* It seems like this is used most commonly when scanning B&W FILM.... don't know the status of this in actual book scanning (or PDFs you find online).

Side Note: I have no way of generating these (GIMP doesn't support it), perhaps Photoshop can export 16-bit grayscales? I will have to hunt around and see if I can find examples of these, maybe I can generate one using Imagemagick. I don't know if ADE could handle these, but it would be fun to toss at it and see what happens (Perhaps it turns into a big black box like my Grayscale + Alpha image).

Indexed PNGs are a replacement of GIFs, and work on ANY colors:

1-bit: 2 colors
2-bit: 4 colors
4-bit: 16 colors
8-bit: 256 colors

1-bit and 8-bit are the most common.
2-bit PNGs seem to have poor/wrong/non-existent implementation in a lot of programs. GIMP seems to output 2-bit and 4-bit correctly.

If the image program is competent, it should select the proper n-bit for the amount of colors.

<= 2 colors should be 1-bit
3-4 colors should be 2-bit
5-16 colors should be 4-bit
17-256 colors should be 8-bit

Side Note: They probably just got lazy and decided to cut down on logic, and only care about 1-bit B/W (think book scans) or 8-bit Grayscale/Indexed. 8-bit encompasses all below, but would just be slightly more bloated than it COULD be.

NOTES on the Test EPUB:

Side Note: Next time, just use zero text appended to the bottom, this means that it will be the SAME EXACT IMAGE, but the only variable thing changing is the different settings. You can always add the explaining text in the EPUB between sets of pictures + relevant filenames.

I used imagemagick's "identify -verbose" command to take a close look:

- Your "2-bit" image is actually saved as a 4-bit image (could be non-existent support for 2-bit PNG in PaintShop Pro).
- Your "3-bit" image is actually a 4-bit image (no such thing as 3-bit lol)
- Your 16->256 is saved as an 8-bit image.
- All the others are saved as the correct bit.

GIMP:

- 2 color image saved properly as 1-bit.
- 4 color image saved properly as 2-bit.
- 16 color image saved properly as 4-bit.
- 256 color image saved as 8-bit.


Testing on my Nook:

- In the 16->256->16 section.
-- The only difference is bit-depth (PaintShopPro = 8-bit, GIMP+ScriptPNG = proper 4-bit).
-- My Nook shows weird artifacts (looks like speckling) on the 4-bit images.



- In the Indexed 8 color section, for some weird reason the GIMP version looks EXTRA HORRENDOUS.
-- Your version + ScriptPNG versions look speckled, but otherwise the same.
-- This is odd because they are all 4-bit, so I would have expected that all three would look horrible... so I have no clue.



- There is weird degradation in your image reduced to 16 colors, there is no degradation in the GIMP 16 color version.
-- This is also odd.

My Image Test EPUB:

OF COURSE I can't get the problem to replicate in my test EPUB consistently NOW... (those gremlins are out to get me lol). But the PNG I was able to get to break was the 2-bit elementof PNG compressed through ScriptPNG:



Here is the problem as I first ran into it in the QJAE (EPUB is attached):



My best guess is... ADE has a bug with 2 and 4-bit PNGs. The "banding" issue is happening on your images, but it is just VERY HARD to tell because of the nature of a photographic image. In my formulas it is much easier to see the bug because there are large swaths of solid color.

Without access to ADE source code, I have no clue exactly what the problem is. I will continue to poke around at the different images with imagemagick... hopefully between comparing all of these different images, I can stumble upon SOME difference. Maybe ADE is ASSUMING 1-bit/8-bit but is ALSO looking at the amount of colors in the PNG, if it falls into those in between ranges, maybe ADE blows up (The amount of colors are saying it should fall into "2-bit/4-bit" while another part of code is forcing it into 8-bit mode?).

This bug COULD have been fixed on a newer version of ADE... could anyone test all of this on a newer e-ink device?

Side Note: Days of work has gone into this. I was reading all this in depth PNG documentation, now time for my brain to explode.

Edit:

Originally Posted by Jellby

That's basically what I mean. With 256->16 you reduce the number of colors to 16, with 16->256 you ensure that the colors are stored as 8-bit numbers.

I just thought of this. Perhaps as a workaround, the image can be reduced to 17 colors instead. This will still force the PNG to 8-bit mode, you will have a very minimal amount of shades of gray, and most of that useless information can be tossed out, leading to a semi-smaller filesize (definitely less than the 256 shades of gray image). ADE SHOULD display that one fine (since it is 8-bit). My brain hurts, I might test it later in the week.

[Jellby]

Originally Posted by Tex2002ans

I just thought of this. Perhaps as a workaround, the image can be reduced to 17 colors instead.

That could work if you don't want to play with the files too much. What I sometimes do is:

GIMP: Reduce number of colors with "Posterize". Depending on the type of image, I reduce to between 8 and 64 colors. Save as PNG.

And then the magic command:

pngcrush -bit_depth 8 -force

to optimize the PNG compression, at the same time enforcing an 8-bit final file.

[SBT]

Ohdearohdearohdear... I've been spewing out loads books with 16 shades of gray PNG images lately. But I've never noticed anything untoward in the images, neither on PC sony Reader or PRS-T1. I use imagemagick (convert in.jpg -colorspace gray -colors 16 out.png), and then use pngcrush, but without specifying bit depth. Maybe that's what prevented the problem.

To be on the safe side, I'll use Tex2002ans' suggestion and use 17 shades from now on...

EDIT: checked the images, the images are indeed 8-bit PseudoClass 256c. The file-sizes are nice and small, though.

[Tex2002ans]

Originally Posted by SBT

Ohdearohdearohdear... I've been spewing out loads books with 16 shades of gray PNG images lately. But I've never noticed anything untoward in the images, neither on PC sony Reader or PRS-T1. I use imagemagick (convert in.jpg -colorspace gray -colors 16 out.png), and then use pngcrush, but without specifying bit depth. Maybe that's what prevented the problem.

What program are you also using to initially create/edit the images? I don't believe it would matter when feeding through imagemagick (since I believe imagemagick unpacks, then repacks the entire PNG), but it might have some effect. Always a good idea to post the image workflow you use as well (thank you for posting the exact imagemagick commands you use).

Attach a few examples, I wouldn't mind testing these on my Nook and see how they display. Also, if you could test the EPUBs from this topic on your device and let us know if you are having the same sorts of problems (or if they are displaying perfectly fine).

It also could also just be that my Nook is running an ancient version of ADE and/or a bug in the specific device firmware. More tests on more devices are always welcome.

Originally Posted by SBT

To be on the safe side, I'll use Tex2002ans' suggestion and use 17 shades from now on...

Well it was just a guess as a way to solve the 2-bit or 4-bit display bug. It will of course need more testing.

Or you could always just continue with the 16 color images, and know that your EPUB should work for compliant/future readers with PROPER minimal filesize.

[Jellby]

Originally Posted by SBT

I use imagemagick (convert in.jpg -colorspace gray -colors 16 out.png), and then use pngcrush, but without specifying bit depth.

Depends on what imagemagick is actually doing. I'd say that "-colors 16" just reduces the number of colors, but does not constrain each of these colors to be 4-bit representable. If that is the case, pngcrush will not reduce the bit depth either.

In short: 4 bits implies 16 gray levels, but 16 gray levels does not imply 4 bits.

[SBT]

Jellby is right, I could've shaved another 6% off the file size if I'd specified 4 bit depth in pngcrush.