Geekmaster's adventures in K1 spelunking

[geekmaster]

Whoa! After going through a stack of microSDHC cards that all failed (0MB size in K1 settings menu), all of a sudden one I had not tested before showed up as 16MB). Cool!

They sure are fussy though (but the K1 mmc module makes a bunch of ASSUMPTIONS about SDHC, rather than trying to read its CID data to see what features it supports). Apparently those assumptions fail on most of my microSDHC cards.

I recall a similar problem a decade ago, when the linux mmc source code contained comments about the guesses it was making about cards, and attempting to access them at a bunch of clock speeds and such -- with web pages compiling lists of which makes and models of SD cards were "linux-compatible". And of course, these K1 drivers ARE a decade old, so really lucky they even WORK AT ALL on a card manufactured in recent years...

As it turns out, this microSDHC card is a 16GB class 2 SanDisk card. Perhaps that "class 2" is important, to prevent the K1 mmc driver from trying to talk to it too fast? Or maybe modern microSDHC cards refuse to operate slow enough for a K1? Perhaps the class (i.e. max write speed) is the important factor on K1 microSDHC compatibility, rather than storage capacity. If so, I need to start collecting (really slow but large capacity) "class 2" cards... I remember fighting this battle when my Palm Pilot (clone) had a max SD card capacity of 1GB, and the smallest the stores sold when I needed one was 2GB. It seems class 2 works (SD or SDHC), but class 4 does not -- and most of my microSD cards are class 10, and they definitely do not work.

[geekmaster]

It looks like I spoke too soon. I just tested a toshiba 16GB card that came with a GearVR (preloaded with software). It WORKS! It is labelled "MicroSDHC, class 4, 16GB, C16G TAIWAN, TOSHIBA".

Hmm... I guess microSD on the K1 is really "hit or miss" (but mostly miss)... Just like was common using SD cards on linux a decade ago, it is mandatory to test specific makes and models of cards to see which work on linux, and that is true for me now BECAUSE the K1 (and its mmc driver) are more than 10 years old.

[geekmaster]

Apparently some class 10 cards CAN fall back to a slow enough speed for the K1. I have a MICRO CENTER 16GB Class 10 SDHC that works. It looks like I need to test cards dedicated to some of my phones and cameras and see what works -- and perhaps do some card swapping.

I would not have bothered with testing all the loose cards (and soon the dedicated installed cards in various devices) if I had not seen the SDHC support (though hard-coded assumptions, probably why so many cards fail) in the lab126-customized K1 kernel drivers...

[geekmaster]

And... just for fun, I stole the 64GB microSDXC (yeah XC, not HC) UHS-1 card from my Galaxy Note 4 (purchased for GearVR use). Of course, it failed. I should feel lucky the K1 didn't catch fire trying to talk to it, eh?

However, the 16GB class-2 microSDHC card I just swiped out of my FPV quadcopter works in the K1 too. So not sure what the limits are. Apparently, any card that CAN work using the hard-coded ASSUMPTIONS in the K1 mmc driver work, and those that don't, don't...

A smarter mmc driver (which reads the specs from the CID table in the SD card) should probably work with all of them. Perhaps even microSDXC cards too, but can you imagine how long it would take to mount a 128GB card if the K1 decided it was time to fsck the card because of some "dirty bit"?

[dhdurgee]

Originally Posted by geekmaster

And being a "bleeding edge" tech collector, I spent many hundreds of dollars years back on a first-generation USB flash stick, a full 128KB (kilobytes) and it still took over an hour to fill it -- USB 2.0 was not invented yet, and USB was originally meant for slow things like mice and keyboards...

But then again, I still have 5MB hard drives in my collection (which hold the equivalent of about THREE IBM floppy disks), and they only moved the read heads (via stepper motors) at TEN steps per second (just like really old floppy drives). It took several seconds to seek from an inner to an outer cylinder. And we thought that was amazingly FAST back then (when most of our code was stored on boxes full of audio cassette tapes).

You certainly bring back the memories! The first computer system I worked on at Drexel was a LINC-8 with 4K of core, dual DEC tapes a teletype console and a 32K Winchester hard drive. We each customized a DEC tape to use, usually having it fail to boot if the switch registers were not set as we desired.

The LINC-8 had both a PDP-8 mode and a LINC-8 mode, one twos complement and the other ones complement. There was a speaker hooked to one bit of one of the LINC registers, so one thing I programmed was a music interpreter taking advantage of it.

I also had access to an APL\360 system, which transitioned to an APL/SV system, via a 2741 with an APL element.

Thanks for the memories.

Dave

[geekmaster]

I still have an ASR-33 TTY (though it would need a major rebuild after all the lubrication turned to an amalgam of dust and tar). Before tha I had my own keypunch machine (and I still have a lot of blank ready-to-use IBM punch cards). And I did keep the APL ROM out of a dual-head Centronics printer before scrapping it. Likewise, I kept a type ball from one of my "networked" Selectric printers. Though I never parted out my daisywheel printer -- it cost TOO DAMN MUCH to scrap. And CP/M wordstar had built-in drivers for it (but it required THREE S-100 Bus printer cards to drive it, as I recall). My list of technical antiquities (many younger than me) goes on and on -- a lot of my gear is bakelite from the "pre-plastic" era, if you can imagine that... I even have a bakelite ViewMaster 3D viewer. And I have NIXIE TUBE calculators and NIXIE TUBE multimeters (19-inch rack mount, of course)... And I used to actually USE this stuff. Kids these days just have no idea how easy life is for them, eh?

EDIT: Hmm... Now I have this strange desire to hook up my daisy wheel printer to a modern-ish dev kit (STM32 perhaps) and make it print. That thing only has about a dozen hours of use on it. Back in the day, people even printed "graphics" by fractional movements of the paper and print mechanisms, while trying desperately to wear out the '.' (period) on the type wheel. --- But that desire will pass...

[knc1]

@dhdurgee:
Search out out "Mainframe in a zipper case" thread if you want to run system-360 software (including vm-370) on your Kindle.

@GeekMaster:
I see to recall you could get an "all ." ('.' on every petal) -
At least I am pretty sure I have one for my Diablo.
http://archive.computerhistory.org/r...4.01.01.lg.JPG

Good point about the 33, -
I should probably also un-box this thing and check the Lubrication/Rust ratio on the (once) moving parts.

Anyone interested in an ADM-3 for their collection of historic "glass tty" devices?

[dhdurgee]

Originally Posted by knc1

@dhdurgee:
Search out out "Mainframe in a zipper case" thread if you want to run system-360 software (including vm-370) on your Kindle.

Not necessary, I have a P/370 here for that.

Dave

[geekmaster]

Originally Posted by knc1

@dhdurgee:
Search out out "Mainframe in a zipper case" thread if you want to run system-360 software (including vm-370) on your Kindle.

Is everything needed in the KUAL extension (second post)?
https://www.mobileread.com/forums/sho...d.php?t=220524

I wonder how hard to port this to the K1...

I found a 16GB class 10 microSD card (actual tested speed 16MB/s write, 19MB/s read), that works in my K1. Really hit or miss finding (non-antique) SDHC cards that fit the hard-coded assumptions in that decade-old mmc driver in the K1.

I will do a "time dd" speed test on this freshly formatted card (using the official "SD Association" formatter) in my K1 now. Then shrink the vfat and add ext3 and swap partitions to it (like I used on OpenWRT aeons ago). I wonder how long to fsck a 10GB jffs2 partition over that (apparently) SLOW K1 SDIO channel?

Anyway, first a terminal app, then Hercules360...

BTW, I had my own personal IBM 360/30 all to myself an entire summer (running in a hotter-than-Hades warehouse). There are cool stories to tell there -- it was a mish-mash of parts, but it got the job done. Can you belief an air-compressor and a punch-ball were integral components of that machine? The large inflated rubber "punch-ball" pressed the mylar punchcards (containing the microcode) against a large array of microswitches). I know because the official IBM service guy swapped in his own personal microcode punchcards whenever he showed up, to run his "proprietary" diagnostics, and he showed me how it all worked inside.

Anyway, "hand-toggled" 360 assembler in a K1 would be cool... ...if only for bragging rights.

[knc1]

Second post is the Hercules emulator, built for the K5.
The first post has the links to the IBM software archives (early mainframe stuff is now public domain).

Still among the missing is the (macro-)assembler for os-360

[geekmaster]

Speed tests. Timing microSDHC on the K1. Running in a script on the K1. And speed-testing the shared drives of the K1 (both shared internal /mnt/us and shared SD /mnt/mmc) using a drive speed tester app running on the host PC.

Whoa! This 16GB class-10 card measured 19MB/s read and 16MB/s write using a USB SD card adapter. But the shared-by-K1 SD card read speed is only 2.5MB/s and write speed TOO SLOW TO MEASURE using this app. Time to measure it differently...

And for comparison, the shared internal K1 flash (/mnt/us) was faster, showing 2.6MB/s reads and about 1.0MB/s writes.

Now, time to measure using that "time dd" script, results posted earlier in this thread (for a class 4 SD card). For this test, the USB MS driver is not loaded (script running in UYK "update install" mode). Much faster SD card, but we shall see if that even matters...

-- Kindle internal storage @ /mnt/us --
Write 100MB/3m42.76s (0.45MB/s)
Read 100MB/1m43.02s (0.97MB/s)

-- 4GB Class 4 SDHC @ /mnt/mmc --
Write 100MB/6m48.61s (0.24MB/s)
Read 100MB/2m48.71s (0.59MB/s)

-- 16GB Class 10 microSDHC @ /mnt/mmc --
Write 100MB/4m43.23s (0.35MB/s)
Read 100MB/2m4.87s (0.80MB/s)

Conclusions? Well... All I can say is that even a fast Class-10 SD card runs slower than the K1 internal flash, and THAT is way slower than USB 2.0 (and in fact even slower than USB 1.0 can support). For that matter, is the K1 a USB 2.0 device?

The Class-10 microSD card is slightly faster than the Class-4 SD card, but not by much.

Perhaps the K1 SD card slot is running internally over USB (and perhaps USB 1.0 at that). The only way to get data on and off the card faster is to highly compress that data. Rather than vfat, jffs2 seems like a much better choice (unless the required jffs2 RAM buffers are too much overhead). Or we could put a really big squashfs partition full of "useful read-only content on the SD card...

Now that I noticed CPU governor logic (available from /proc files) in the kernel source code, I wonder if we can make SD card writes faster by futzing with the CPU governor? The default appears to be half-speed, but it can go MUCH slower (by a factor of 1000). Flipping it to full-speed (if it does not do that automagically as-needed already) might make this test internal SD card run faster (but still not much faster than the internal storage speed, probably).

EDIT: One thing interesting is that when the K1 copies a 100mb file from its internal /mnt/us drive to /dev/null, it runs at only about one-third the speed that the USB mass storage driver shares a file to a host PC. Is /dev/null really THAT SLOW? I suspect a "CPU governor" thing in play here, where USB mass storage pumps up the speed so as not to annoy kindle owners who side-load stuff onto their kindles... If so, we might be able to increase internal speed as well, by making our own adjustments to the CPU governor (as mentioned above).

[geekmaster]

Just so I can use that Class-10 microSD card in the camera for which it was purchased (needed to take photos WHILE filming 1080p video), I am now speed testing my 16GB Class-2 microSD card. The class rating in the MINIMUM write speed, but cards are often faster (downmarked as the market demands).

This Class-2 microSD card measured 5MB/s write and 22MB/sec read speed, faster than some of my Class-4 SD cards. And its read speed is even faster than that class-10 card measured above (though writes are slower).

Really though, in the end, the only thing that will matter to improve storage speed is COMPRESSION (i.e. squashfs or jffs2) and I was hoping for RAM compression too (zram, or compcache if I need old code for K1 compatibility). The I/O channels on the K1 are just slow, so it is something to work around however we can... But then, newer kindles might not be all that much faster either (speed tests needed).

[geekmaster]

While hunting for my 16GB Wifi SD card (hacked so I can SSH into its internal processor), I found another bag of microSDHC cards. I really do not feel like testing them for K1 compatibility. I have enough for that purpose now.

The reason I want that Wifi SD card is that I can SSH in over wifi, and it can read and write to the same SD storage space as the kindle. There are still the problems with potential filesystem corruption (whoever writes a given block last wins). But as I mentioned, I should be safe with pre-allocated files, where both sides have complete control over who gets to write where (within that chosen file or files). That will let me run a rudimentary "shared file" remote shell (which I have already tested between a VM and its host PC). The SD card processor and the kindle do not have any other means of communication other than (potentially risky) shared files, but with careful management that should be fine.

That is a quick way to gain remote shell, until I manage to compile all dependencies for usbnet access. The only other alternative its corkscrewing a shell over whispernet, and as I said before, that is a bad idea for multiple reasons.

Anyway, the next best option until I find that wifi card (new purchases are off-the-table at the moment) is an on-device terminal, now that I know all the K1 keycodes.

[geekmaster]

I extracted the root filesystem squashfs image file from the amazon FW 1.2 download for K1. I copied this squashfs image file to my SD card. I created a mountpoint dir on the SD card. I successfully loop mounted the squashfs rootfs-1.2 image file, and my RUNME.sh script (launched from a UYK update.bin) did a directory listing for the inner contents of that mounted rootfs, right after mounting it, so it seems to work fine.

Be sure to umount (UNMOUNT) everything on the SD card before exiting, or the kindle GUI framework will complain about a "locked SD card" and refuse to mount it. Likewise, a "busy" SD card does not get exported as a shared USB drive.

Also, a short delay before exiting seems to help, because exiting immediately after unmounting the loop mount sometimes still gets reported as a locked SD card.

Now I am curious how many books I could fit into a highly-compressed squashfs partition on my 16GB microSD card (which works fine in my K1). I believe the kindle should see the books if my mount point is in the "directory" folder where you store your books. I will try that RSN... But I will need to do the loopmount in a startup script AFTER the SD card is mounted, to prevent that "locked card" warning.

One problem though -- it is taking intolerably long for the kindle to exit USB MS (USB drive) mode with my SD card in, because both sides (the kindle AND the host PC) seem to think they need to do a filesystem check (almost) EVERY TIME, even though I am properly ejecting the devices (and waiting awhile before disconnecting USB cables).

One time recently, my PC decided to move ALL the files on my kindle (/mnt/us) into a FOUND.000 directory, renamed as .CHK files (and extended to a multiple of 4K). So I am more than a little wary of LETTING it do that filesystem check it wants to do all too often. It could be that I just need to reboot my host (Windows) PC. Weird crap starts happening if you go too many days without a reboot...

When the K1 is doing a filesystem check on the SD card (after disconnecting the USB cable), it takes nearly a minute before it even starts the "busy spinner" animation, and that is rather hesitant and herky-jerky for awhile before it even performs good spinner "animation".

Perhaps a smaller SD card would be faster to mount and dismount? Or better yet, stop using USB MS altogether and stick with USBnet (and SCP for file transfer). Of course, I need to add USBnet support (missing kernel module) for that...

FYI, the squashfs image supplied in the latest amazon K1 firmware (1.2) is squashfs 3.0, which I cannot mount on a recent linux (format too old, no longer supported). However, it mounts just fine in the K1. I noticed that other partitions in the K1 flash module are also stored in squashfs (read-only) format -- only /opt and /mnt/us are writable ext3 formats.

[knc1]

See:
http://squashfs.sourceforge.net/

Heading:
"Releases"

Download page:
https://sourceforge.net/projects/squ...iles/squashfs/