M5Paper e-ink device for DIY

[stevenaleach]

Well, the e-ink shipping race is on. I ordered "the good" Waveshare display today: "Waveshare 10.3inch Flexible E-Ink Display HAT for Raspberry Pi 1872×1404 Resolution with Embedded Controller IT8951 Communicating via USB/SPI/I80/I2C" and it's due sometime in early April... as is the M5Paper.

Meanwhile, I've got (nearly) all components collected and mostly soldered and tested for the waveshare-reader build: a 400x240 "Sharp Memory Display", a 512k FRAM SPI breakout, and an I2C GPIO expander, along with two push-button/rotary-encoder knobs, and a tiny 1200mah battery.

The gpio expander might be a mistake: there's enough open pins that I could read the knobs without it, but 1) it has an interrupt pin which is nice and elegant since I can have any input action (knob press or twist) trigger wake-up on a single pin... which is dern convenient, and 2) it provides greater flexibility later by minimizing the pins used on the board, using just I2C and a single extra pin for the interrupt. But it has an obnoxious power wasting LED, which I didn't expect and don't like... I hate to cut into battery life just because I wanted to make the code a little shorter and simpler and keep pins free that I may or may not ever use... hmm... but for now it stays.

The FRAM is added for non-volatile storage with near-infinite rewrites - someplace to store current page numbers for current and recent documents, any data that needs to survive reboots and that changes often to avoid wearing out SD cards or the internal flash.

Tomorrow's project is wiring up and figuring out how to interpret those rotary encoders, and then to start working on a serial shell for transferring and managing files as well as more interesting stuff like receiving and displaying bitmaps in real-time so the device can act as a nifty display panel while it's connected to the laptop.

The state of the build, thus far:
https://pasteboard.co/JRtH5ZX.jpg

[stevenaleach]

Note the battery connection... I don't know whether it's the battery (cheap one off of Amazon), or the Lolin board... but someone got the wires wrong... It didn't seem like the battery/charging circuit was working at first, and since there's only one way the connector can possibly connect it took me a couple days to notice that the board had a '+' printed next to the side of the connector with the black-lead from the battery... checking with the multi-meter verified that, assuming this is a standard connector, one of the two manufacturers got it wrong. After reconnecting appropriately (red to black and black to red), it ran from the battery... huh.. apparently the battery, board, or both have good protective circuits that prevented damage from the previously reversed connection. Nice.

[Quoth]

Link doesn't work and has loads of obnoxious javascript.

[hobnail]

Originally Posted by stevenaleach

Note the battery connection... I don't know whether it's the battery (cheap one off of Amazon), or the Lolin board... but someone got the wires wrong... It didn't seem like the battery/charging circuit was working at first, and since there's only one way the connector can possibly connect it took me a couple days to notice that the board had a '+' printed next to the side of the connector with the black-lead from the battery... checking with the multi-meter verified that, assuming this is a standard connector, one of the two manufacturers got it wrong. After reconnecting appropriately (red to black and black to red), it ran from the battery... huh.. apparently the battery, board, or both have good protective circuits that prevented damage from the previously reversed connection. Nice.

A youtube video I watched showed the same problem with the reversed leads.

[stevenaleach]

Still no updated information on the long-awaited and still long awaiting M5Paper... Amazon, meanwhile, has sent me a notification on the Waveshare display, which is shown to have shipped... and is expected to arrive between March 26th and April 16th. Grr... Arrg.

Meanwhile, the mess of wires I've been soldering together here is ready to connect to the display and start testing/coding as soon as it gets here. I gave up on the GPIO expander - it was probably just a bad connection somewhere that multiple rounds of checking and re-soldering failed to cure, but the board kept resetting and glitching on me and acting generally flaky when touched... so the esp32 is handling the rotary encoders directly - which leaves me with *just* enough open pins to connect the Waveshare display and no real room for future expansion after that, but that's fine, and it'll use just a tad less juice since it eliminates that irritating 'active' LED. I've got two rotary encoders and the little memory display for user-interface, the FRAM chip and 32GB SD for storage, and everything seems to coexist and work happily together. My one gripe is that I have to use software SPI for everything (SD, memory display, FRAM and, eventually, the eInk display as well) because if I use hardware SPI for any other device, the memory display quits working... that's a real pain because It doesn't appear I can use software SPI with the standard Arduino/ESP32 SD libraries - I found one that works, https://github.com/nhatuan84/esp32-micro-sdcard, but has some irritating limitations like eight character filenames with three character extensions, but that's not a huge problem.

I thought I was joking when I said I'd probably have the DIY reader finished before the M5Paper finally reaches my mailbox, but it's looking more and more like that will actually be the case -- though I probably won't have it in a frame/case yet since I can't get a head-start on that - I need the screen and its controller in hand so I can measure and ponder and probably wander the shelves of the hardware store and thrift-store a bit (I expect I'll wind up using a picture frame, I don't know yet) .... but I'll probably have actual images and text displayed on a lovely e-ink screen from this thing before the M5Paper gets here.

[hobnail]

Originally Posted by stevenaleach

My one gripe is that I have to use software SPI for everything (SD, memory display, FRAM and, eventually, the eInk display as well) because if I use hardware SPI for any other device, the memory display quits working... that's a real pain because It doesn't appear I can use software SPI with the standard Arduino/ESP32 SD libraries

The Arduino API is a layer on top of (wrapper around) the ESP-IDF API.

Several times I've read that the Arduino libraries have limitations that aren't there when you use the ESP-IDF so I'm focusing on the latter.

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/

[stevenaleach]

Originally Posted by hobnail

The Arduino API is a layer on top of (wrapper around) the ESP-IDF API.

Several times I've read that the Arduino libraries have limitations that aren't there when you use the ESP-IDF so I'm focusing on the latter.

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/

I think the memory display is the problem - it's a write-only device that doesn't use the MISO pin, the SD and FRAM work fine with hardware SPI - but the display no longer works. If I stick with software SPI for everything, then it's fine. Not a problem because the IT8951 code I've looked at all supports software SPI and though hardware SPI would be a bit faster, it won't really matter. Menu scrolling and UI interaction on the little screen will still feel "instant" -- It's the e-ink screen that will suffer the most from the slower transfer rate of software versus hardware SPI, but it's also just slow to begin with - taking "< 1 second" according to WaveShare (so we'll assume a good fraction of a second at least) to refresh. Sending a whole page at once, the slower SPI transfer rate might add a few milliseconds to that. If partial refresh scanning down the page transferring more data while the previous chunk is being drawn is possible then it'll make no difference at all - but I don't know if the IT8951 is that clever or not, I haven't dug that deep into it yet and don't yet have one on hand.

I'm VERY pleased with the power usage - I'd bought the cheapest little battery I could find that looked like it would do for prototyping (3.7V 1200mAh) - I expected, however, that after I knew how much room I'd have in the eventual case, I'd get the biggest battery I could fit because I want to be able to get several days, if not some weeks out of a charge. Well... I had some test code running for at least six hours yesterday that continuously polled the encoder pins, updating the display immediately on any changing, and updated the display every second if there weren't changes, displaying battery voltage... after over six hours, it had *just* dropped below four volts.... considering that when in use as a reader, the controller will spend most of its time sleeping and the little display will be off, this little battery probably will be good for a week with 3.0V as a nice safe cutoff point. Especially since the FRAM will be holding the 'restart' information instead of (or in addition to) RTC memory which means I'll be able to add a toggle switch on the battery line - that way it can just be shut OFF when it's set down, or when it's displaying a page that won't be changing for a while (like a piece of sheet music) and doesn't need to be powered on.

So no need for the big battery - which is especially cool because the one I've got weighs approximately nothing and I went with the flexible e-ink display instead of glass because I'm chicken and decided the flexible panel that wouldn't shatter from eventual drops was worth the extra cost. I wasn't thinking about or prioritizing weight but it seems this reader/display gadget is going to be remarkably light.

[stevenaleach]

Just 'finished' a tool that others mixing micro-controllers and ePaper might find useful:
https://github.com/stevenaleach/PDFto4BC It.s a Python tool to convert .PDF files into a form more friendly to low power low memory devices. Pages are run-length encoded pixel-pairs (two pixels per byte) and the compression seems to average at about 14% of what the raw (1 byte per pixel) pixmaps would take up. I'm converting to 1404x1872 because that's the size of the Waveshare display I'm waiting on. The first full book I converted was Stephen Levy's 'Hackers' - 520 pages that would occupy 1.366 GB as uncompressed pixmaps or 683.3MB as packed pixels, two per byte -- the run-length compression brings this down to ~170MB with no cropping applied - 215MB with nice cropping to nearly eliminate margins. Not too bad - at that ratio I can store nearly 2,500 pages per GB - and some texts compress better - I've been testing on several .PDFs of Creative Commons licensed books, and when I ran the file I have for Charles Stross's book 'Accelerando' (819 pages), it came out at 186MB -- or 4,500 pages per GB.... fonts matter... text with thicker and softer fonts will compress less than those with thin and sharp edged fonts... and of course images don't compress well (the cover of Hackers, for instance squeezes down to about 34% of the 'raw'' size while the pages are about 14%).

https://github.com/stevenaleach/PDFto4BC

[Quoth]

Obviously you mean PDFs that are envelopes for images and not ones that can can have the text extracted and converted to something more CPU and screen friendly?
How does it compare with greyscale and 1 bit monochrome multipage TIFF, which used to be more common for scanners and electronic Fax reception before PDFs were misused for the purpose?

[stevenaleach]

Originally Posted by Quoth

Obviously you mean PDFs that are envelopes for images and not ones that can can have the text extracted and converted to something more CPU and screen friendly?
How does it compare with greyscale and 1 bit monochrome multipage TIFF, which used to be more common for scanners and electronic Fax reception before PDFs were misused for the purpose?

Not sure what you mean - it works best on text pages, not images obviously, since the vector fonts resize beautifully with no pixelation - that's kinda the whole point. Pages with images or pages that *are* images will be resized as well but of course there will be some pixelation.

[Quoth]

Text "dictionary compresses" well. It should be stored as text and rendered on the fly. HTML or Postscript can be tokenised.
Colour and 8 bit grey images should probably be pre-converted to 1 bit, 3 bit or 4 bit mono.

[stevenaleach]

Originally Posted by Quoth

Obviously you mean PDFs that are envelopes for images and not ones that can can have the text extracted and converted to something more CPU and screen friendly?
How does it compare with greyscale and 1 bit monochrome multipage TIFF, which used to be more common for scanners and electronic Fax reception before PDFs were misused for the purpose?

I'd originally expected to use plain-text/markdown for a reflowable format for the device and only convert .PDFs for technical documents, and etc. where I wanted to preserve the original formatting and rendering, but I think now that I'll skip the plain-text entirely -- converting and sending .PDFs to the device really achieves all I need and compression is good enough that I don't need to save space by storing plaintext instead.

Plus, reflowable text would mean having rendered characters in a set of sizes in a set of fixed-width fonts (at least a serif and non-serif in regular, bold, italic, and bold-italic) -- a route I'd started to go down, generating three-bit (to free a bit as a run-length repeat flag) packed-pixel font pixmaps... but if I were to cover all the characters I'd like to support (basic and extend Latin, Greek characters, mathematical symbols and operators) in a range of sizes useful on the e-ink display... well... yikes... that's a lot of data just for fonts.

It turns out that just doing the heavy lifting and rendering on the computer instead is a whole lot easier. And the results are nicer - if I start with a .PDF of the text I want that's nicely laid out, the results will look great on the screen.

The format I'm using is designed around offering reasonably good compression, while still being extremely easy for a microcontroller to interpret - the bit depth is always and only four-bits per pixel, all pages are the same dimensions, and the pages are fast and easy to decompress after determining the page's offset into the file from the header.

[murraypaul]

Originally Posted by stevenaleach

Plus, reflowable text would mean having rendered characters in a set of sizes in a set of fixed-width fonts (at least a serif and non-serif in regular, bold, italic, and bold-italic) -- a route I'd started to go down, generating three-bit (to free a bit as a run-length repeat flag) packed-pixel font pixmaps... but if I were to cover all the characters I'd like to support (basic and extend Latin, Greek characters, mathematical symbols and operators) in a range of sizes useful on the e-ink display... well... yikes... that's a lot of data just for fonts.

I think you are working more directly with the hardware, rather than using the M5 development stack?
Because the board API they supply already has code for the device to render arbitrary TTF fonts at different sizes and display them. They don't have to be fixed-width either.
I have a toy-working text + basic bold/italic display with full justification with non-fixed-width fonts, the API does most of the work already.

[stevenaleach]

Originally Posted by murraypaul

I think you are working more directly with the hardware, rather than using the M5 development stack?
Because the board API they supply already has code for the device to render arbitrary TTF fonts at different sizes and display them. They don't have to be fixed-width either.
I have a toy-working text + basic bold/italic display with full justification with non-fixed-width fonts, the API does most of the work already.

And the Adafruit GFX library provides (mono-spaced, scalable - though only by multiples of the base size bitmap) font support as well - and is what I'll be using for printing to the little "interface" screen on the device - an Adafruit SHARP Memory Display - it's what the memory display library is built on/with -- there's a Waveshare library built on Adafruit GFX as well, but it doesn't yet support the display I ordered and I don't yet know how hard it would be to modify -- but I'll probably skip that and go with minimal code based on Waveshare's examples on Github that just implements a full page update.

The big 1404x1872 e-ink display will be used as a page canvas only (never being used for menus/UI display, which is why I really don't need fancy stuff like partial updates). I *could* print text on that canvas and basically duplicate the traditional ereader experience (when used to view EPUB/MOBI) - but that wouldn't really justify the nice display..

But by using compressed pixmaps and a simple container format, I can render the page on the computer from a .PDF file that looks *exactly* how I want. The first book I converted, as I said, was 'Hackers' by Steven Levy. One of my favorite books as a kid, and I can honestly say, a text without which I probably never would have wound up soldering together an ereader thirty years later (or be quite as strange in general),.... now the .PDF that Google served up to me is *EXACTLY* the same as the print version, and it feels like the book I know and love and have owned several physical copies of over the years. The second book I converted was K&R's "The C Programming Language, 2nd Edition". Now again, this is a text I've owned many copies of through the years, a text I know and love, and the .PDF is exactly the same presentation as the printed text - the way it "should" be.

Now your argument is that these documents "should" be handled as text instead... well... there's plenty of affordable e-readers on the market that can display .MOBI/.EPUB/plain-text... but I wanted something that I could actually put books on... not just the text content of the book, but the actual pages as they should be... and for technical documents, it's absolutely essential - think Arxiv papers with graphs, figures, text associated with figures, equations, etc. etc... There's at least a few very-much-not-affordable (>$600, some in the thousands) e-ink tablets that can handle .PDFs.... but even if the cost weren't a concern all of these are too-much device - lots of extra features I don't want, don't offer battery life I'd be happy with, take time to boot up a full OS, and and general aren't what I want -- and none could be taught other tricks like acting as a USB display to plot/'print' things to from the laptop... so the DIY route it is.

Text that isn't in .PDF form yet.... that's what Pandoc is for ;-) If I want to read a book in EPUB/MOBI format, I can convert it using Pandoc, playing around until I find a font and other options that I like, and then I can convert the .PDF and load it onto the device.

And just imagine trying to implement Arduino/Espressif code to handle Latex Math in the available memory.... and if you COULD pull off that superhuman feat, how long would it take to draw a page with a lot of formulas? Yikes... But why would I want to do anything like that, when I can let Latex handle the typesetting for me and generate a nice .PDF ready to be converted and loaded on the device.... That's a lot more straight-forward.

Or sheet music.. Yea... I COULD come up with a markup scheme for chords, chord-notes, melody notes, and draw a score on the screen --- or I could use LilyPond if I want to typeset a score from scratch and it'll produce an absolutely beautiful .PDF. ... and my collection of 'Real'/'Fake'-book leadsheet collections that are, of course, all in .PDF form... sure I *could* figure out a way to draw the staff and some sort of markup scheme and then I could treat it all as 'text' (after typing it all in)... but... well... I'd most certainly rather not ;-)

The only thing I wanted to change from the "print" pages was the margins - since the screen will be surrounded by a bezel/matting, the visual effect of the original margin will be replaced and I want the "ink" brought as close to the edges of the display as possible, while leaving cover pages and similar with full-page artwork alone - all of which that script takes care of.

While MOBI/EPUB type formats were appropriate and necessary when the first Kindle came out (with only 250MB storage, and a 800x600 black and white screen), I'm using the cast-off 32GB SD card that used to live in my phone (until my music collection *finally* got too big for it), and could upgrade it so cheap if storage becomes an issue that I'm absolutely not worried about storage space -- 150MB to 200MB for a book around 500 pages at the absurd resolution of the big Waveshare screen is just fine.

For the M5Paper that I'm also... stilll.... waiting to arrive at my door, it's a bit of a toss-up - since the smaller screen might be too darn small, converting .PDFs to display on it probably isn't as attractive of an option, I won't be able to get the same experience for most books on the M5Paper of having it look and feel "just like the paper version", But really I doubt if I'll even be using the M5 as a reader much - I expect it will be spending most of its time as a bluetooth macro-keyboard and password manager along with various temporary display and control panel duties every now and then - it'll probably always just be the thingamajig that I will usually be able to reflash into something else periodically without losing anything important. Also, the M5Paper has a touchscreen -- which means that actually interacting with text on the screen and entering search terms becomes an option, making having text as text rather than mysterious and unsearchable pixmaps much much more attractive.

That won't be the case with my DIY reader which will have two 'knobs' for a user interface - one to select and switch between files, and one to turn the page or jump to a desired page... no keyboard, no touchscreen. It will be very much a single-purpose device (well... two purpose device), acting essentially as an alternative to a printer and physical bookshelf/filing cabinet full of 'printed' documents. It's second role will be acting as a USB connected display through a 'receive and display' command in the serial shell - which will take a single raw pixmap and throw it on the screen -- that'll be nice... I already have a second monitor turned to portrait orientation next to my laptop that a Jupyter-Lab session always lives on -- I can't wait to be able to park the e-ink screen next to that to send plots to, to display reference material on, to preview 'printed' pages on, etc.

[stevenaleach]

Guess what I found on my mailbox today..... :-) !!!!

Just a mockup for now, It arrived sooner than Amazon's most optimistic estimate, so it turns out I'm not quite ready to solder yet.... I've got to get a serial shell up and running with a few basic file transfer commands, and the UI up and running on the device first - which I'd figured I had a couple more weeks to get done... nice to be wrong about that ;-). I'll start soldering again as soon as I've got a text or two on the device and the UI working nicely to switch between documents and pages, and to load pixmaps of the desired page into a display buffer. Then I'll be ready to connect the e-Ink display and start figuring out how to talk to the driver board and put pixels on the screen... until then, I don't want to risk damaging it -- Waveshare warns everywhere that the little ribbon cable is fragile.... so the less handling the better until everything else is done and the Knoblet is silently singing "If I Only Had A Screen...".



https://gateway.ipfs.io/ipfs/Qmabku8...VtV/mockup.jpg

EDIT: Link finally works... but the board still doesn't seem to like it as an image tag... too big?.. Oh well.. **... Oh... There we go... it works.