Firmware 4.31.19086

[MGlitch]

Originally Posted by davidfor

I don't think so. The reports of the scan prompt come from Windows users, but, that is because that is what Windows does. Linux is much less likely to report the filesystem as dirty. I don't know if Linux is less aggressive with this, or it does other checks, or not.

If the issue is causing a corrupt database, that will be detected on an OS when the database is accessed. Calibre will do that, or you can use an SQLite database management tool or command-line tools to check.

Still if it were a Kobo bug you’d think we’d see much more reports of it, especially here since I’d estimate the majority of us side load. Granted as you say it’s not every time that it happens but few enough people seem to have corrupted databases that it might point to something outside of Kobo. The scan and fix message has plagued windows users for years now.

I’m curious if those having issues eject and unplug when the OS reports it safe or unplug when the Kobo goes back to the homescreen as these events aren’t always in sync with each other.

[davidfor]

Originally Posted by MGlitch

Still if it were a Kobo bug you’d think we’d see much more reports of it, especially here since I’d estimate the majority of us side load. Granted as you say it’s not every time that it happens but few enough people seem to have corrupted databases that it might point to something outside of Kobo. The scan and fix message has plagued windows users for years now.

I’m curious if those having issues eject and unplug when the OS reports it safe or unplug when the Kobo goes back to the homescreen as these events aren’t always in sync with each other.

I'm not sure if I've said it before, but, my first thought on the problem is the font issues in 4.31. For me, the scan-prompt seems to be connected to adding an epub. They are imported using the RMSDK. And the cover images are rendered using the RMSDK. That means the fonts are being loaded. The way the fonts have been loaded was changed in 4.31 to try to solve an issue (you get a fallback rather than a crash). If the people are seeing this all the time use sideloaded fonts and have selected one of them as their default font, it could be the cause. Me, I use the publisher defaults, and the only reason I have sideloaded any fonts was to test the font problem.

The other thing we don't know is if they use other things such as:

• Patches
• NickelMenu
• NickelSeries
• KOReader
• NanoClock

Or other things. Some of those will be loaded or active during the import process. I don't use them and I don't know how they interact at the times when the issues seem to happen. But, if someone is having these problems and is using them, then testing without is needed to prove it isn't them.

[NiLuJe]

That's probably a sensible guess.

FWIW, what I did the last time (after first noticing the issue outside of Nickel entirely), is simply to enable the force wifi flag, and then to look at what's happening before/during/after the USBMS session in an SSH shell (... one that doesn't have its claws on onboard itself, though ^^), mostly by poking at dmesg, the syslog, the procfs tree and lsof.
(Nickel does a lazy unmount, which means you'll see a whole lot of fds pointing to / instead of /mnt/onboard, since that's how the magic happens ).

Another thing that changed on 4.31: all the kernels have been updated, and I don't have the source delta to tell you how exactly, except for a relevant bit on mx6sll & mx6ull devices, where most of the USBMS machinery is now built-in to the kernel, instead of being pluggable modules (this *will* affect timings, which is a sensible issue on NTX boards ^^).

Patches shouldn't have any impact on fs handles; nor should NM (barring direct user interaction with NM and very unlucky timing, and even then, highly unlikely).

KOReader/Plato won't have any impact on this, since they're completely inert inside Nickel. Restarting Nickel after a KOReader/Plato session *might* exacerbate/uncover weird issues though, but nothing on a cold boot.

NanoClock is a very sneaky piece of trickery, so I can't honestly say no, although I wouldn't think it would affect these sorts of things (i.e., on some devices, it's more likely to crash the kernel during the import progress bar than anything else ;p).

NickelSeries interacts directly with the import process, so, who knows, but, much like NickelMenu, it's by definition bound by Nickel's own constraints.

[rashkae]

Originally Posted by davidfor

I'm not sure if I've said it before, but, my first thought on the problem is the font issues in 4.31. For me, the scan-prompt seems to be connected to adding an epub. They are imported using the RMSDK.

I was able to replicate the issue 3 times in a row without adding *any* books. (and all the books I have loaded are Kepub)

I have installed none of the patches. The only thing Unusual I did was enable the new sideload mode.

When the Kobo is connected to my system, I see a "File system error, Run repair" error in dmesg. (I'm paraphrasing from memory.) The kobo ejects without problem and seems to be working. However, if I start reading a book, then power it off, (from within the book.),, when I turn it back on, my reading position was not updated. If the kobo is powered off then back on after being connected to PC, there is no issue with reading position not saving on following power off. None of these problems are present after downgrading to 4.30

Edit: It's worth noting that I had been using the device since the firmware was first released without noticing any problem. It wasn't until I went looking for it that I was even aware there was an issue. I generally don't power off my kobo, (only sleep and wake.)

[martinot]

Originally Posted by Poppaea

I asked because the Aura One had similiar low battery run times in the beginning and they were solved with an update.

The Sage, Forma and One do have the same size battery, so I guess reading only should give similar battery times, unless the faster processor sucks up a lot.

As I understood it, the new CPUs consume more than the previous one in A1 and Forma. I think that is the reason they increased the battery in Libra 2 to 1,500mAh.

[frostschutz]

Just tested USB connection on Libra 2, 4.31 firmware, no mods installed (other than dev mode enabled for telnet access).

Umount always fails when you connect within KEPUB (nickel does not release KEPUB filehandle), and sometimes fails when you connect from Home menu (nickel does not release KoboReader.sqlite filehandle).

Umount works well from within regular EPUB book. So if you have USB corruption issues I recommend you open a regular EPUB first before making the connection, and see if that helps.

Lazy umount is a bug that helps sweep other bugs under the rug, so no one will ever notice or fix them.


----

if anyone wants to test, put the following code in /usr/local/Kobo/udev/usb just *before* sync umount -l /mnt/onboard

it will try to umount regularly 5 times, if it fails you'll find umount.txt in the exported filesystem showing a process list and open filehandles and such

        attempts=5
        for i in $(seq $attempts)
        do
            grep " /mnt/onboard " /proc/mounts || break
            umount /mnt/onboard && break
            (
                echo "-------- umount failed ($i / $attempts) --------"
                umount /mnt/onboard 2>&1
                date -Is
                pstree -p
                grep -E '^([^ ]+ ){3}b3:0[39] ' /proc/*/maps
                find /proc/[0-9]*/cwd /proc/[0-9]*/fd -exec stat -tL {} + | grep -E '^([^ ]+ ){6}b30[39]' | xargs ls -l
            ) > /tmp/umount.txt
            cat /tmp/umount.txt >> /mnt/onboard/umount.txt
            sleep 1
        done

[Sirtel]

Originally Posted by martinot

As I understood it, the new CPUs consume more than the previous one in A1 and Forma. I think that is the reason they increased the battery in Libra 2 to 1,500mAh.

The Libra 2 has an old CPU. So I wonder why they did it there - for audiobooks, perhaps?

[icallaci]

Originally Posted by MGlitch

I’m curious if those having issues eject and unplug when the OS reports it safe or unplug when the Kobo goes back to the homescreen as these events aren’t always in sync with each other.

I always eject my Elipsa and wait for the OS to tell me it's "safe" to disconnect it. I never just yank the cable. BTW, I'm on Windows 11.

[ownedbycats]

Originally Posted by frostschutz

Umount always fails when you connect within KEPUB (nickel does not release KEPUB filehandle), and sometimes fails when you connect from Home menu (nickel does not release KoboReader.sqlite filehandle).

Huh. That's something I actually didn't consider.

Sometimes I connect from the home screen and sometimes I connect while I'm reading a book. I do always make sure to switch back to home menu if I'm sending a new version of the book I'm currently reading (e.g. in-progress fanfics).

Originally Posted by frostschutz

Umount works well from within regular EPUB book. So if you have USB corruption issues I recommend you open a regular EPUB first before making the connection, and see if that helps.

Unfortunately, since I use KoboTouchExtended since before I got my H2O, I think... well, every book I have on it is a kepub. Though I guess I could manually copy some random ePub onto my device.

[davidfor]

Originally Posted by frostschutz

Just tested USB connection on Libra 2, 4.31 firmware, no mods installed (other than dev mode enabled for telnet access).

Umount always fails when you connect within KEPUB (nickel does not release KEPUB filehandle), and sometimes fails when you connect from Home menu (nickel does not release KoboReader.sqlite filehandle).

Umount works well from within regular EPUB book. So if you have USB corruption issues I recommend you open a regular EPUB first before making the connection, and see if that helps.

I'll have to watch for that. I just plugin and connect no matter whether a book is opened or not. In fact, I frequently have the book opened showing the annotations list after fixing errors in the book. I clear the list and plugin.

I'm mixing epubs and kepubs, so that might explain some of what I am seeing.

Lazy umount is a bug that helps sweep other bugs under the rug, so no one will ever notice or fix them.


----

if anyone wants to test, put the following code in /usr/local/Kobo/udev/usb just *before* sync umount -l /mnt/onboard

it will try to umount regularly 5 times, if it fails you'll find umount.txt in the exported filesystem showing a process list and open filehandles and such

        attempts=5
        for i in $(seq $attempts)
        do
            grep " /mnt/onboard " /proc/mounts || break
            umount /mnt/onboard && break
            (
                echo "-------- umount failed ($i / $attempts) --------"
                umount /mnt/onboard 2>&1
                date -Is
                pstree -p
                grep -E '^([^ ]+ ){3}b3:0[39] ' /proc/*/maps
                find /proc/[0-9]*/cwd /proc/[0-9]*/fd -exec stat -tL {} + | grep -E '^([^ ]+ ){6}b30[39]' | xargs ls -l
            ) > /tmp/umount.txt
            cat /tmp/umount.txt >> /mnt/onboard/umount.txt
            sleep 1
        done

Strange you should mention that. I was hunting through the firmware looking for changes. A few lines below that code, there is a change. In 4.30, it looks like:

		if [ $CPU == mx6sll ] || [ $CPU == mx6ull ]; then
			PARAMS="idVendor=$VENDOR_ID idProduct=$PRODUCT_ID iManufacturer=Kobo iProduct=eReader-$VERSION iSerialNumber=$SN"
			/sbin/insmod /drivers/$PLATFORM/usb/gadget/configfs.ko
			/sbin/insmod /drivers/$PLATFORM/usb/gadget/libcomposite.ko
			/sbin/insmod /drivers/$PLATFORM/usb/gadget/usb_f_mass_storage.ko
		else

And in 4.31:

		if [ $CPU == mx6sll ] || [ $CPU == mx6ull ]; then
			PARAMS="idVendor=$VENDOR_ID idProduct=$PRODUCT_ID iManufacturer=Kobo iProduct=eReader-$VERSION iSerialNumber=$SN"
			/sbin/insmod /drivers/$PLATFORM/usb/gadget/configfs.ko && /sbin/insmod /drivers/$PLATFORM/usb/gadget/libcomposite.ko && /sbin/insmod /drivers/$PLATFORM/usb/gadget/usb_f_mass_storage.ko
		else

My shell scripting is terrible. But, in the 4.31 version, I think that means if one of the commands fails, the later commands are not executed. Whereas in the 4.30 version, they will all execute no matter what. They shouldn't, but, it's the only thing I can see that might be involved. But, it's probably more why they fail, and the umount failing might explain it. Of course, my tendency would be to think if these commands didn't work, that the connection could have more problems than what we are seeing.

Hopefully someone with better knowledge on this sort of thing can comment and possibly test.

[compurandom]

Originally Posted by davidfor

Hopefully someone with better knowledge on this sort of thing can comment and possibly test.

The 30 version you listed always tries to load all the drivers.
The 31 version only loads successive drivers if the previous one didn't fail.

[ownedbycats]

Can I just mention a moment of appreciation for everyone on MobileRead?

[compurandom]

Originally Posted by MGlitch

Still if it were a Kobo bug you’d think we’d see much more reports of it,

Well, consider this a "much more report".

And I'll report that this is an issue from before 30, let alone 31.
It just isn't as frequent.

I've corrupted my database multiple times with calibre using send to device / eject / import / reconnect cycles in short order.

I think the first time I corrupted it, I thought I had maybe not ejected correctly once or twice so I didn't report it. Now I strongly suspect I didn't miss any steps and something was just holding the mount after an import.

Note that I found it is much less likely if I only do one cycle per day, which leads me to believe that something is taking its sweet time to finish after the import completes.

[ownedbycats]

Originally Posted by compurandom

I've corrupted my database multiple times with calibre using send to device / eject / import / reconnect cycles in short order.

I think the first time I corrupted it, I thought I had maybe not ejected correctly once or twice so I didn't report it. Now I strongly suspect I didn't miss any steps and something was just holding the mount after an import.

Note that I found it is much less likely if I only do one cycle per day, which leads me to believe that something is taking its sweet time to finish after the import completes.

That also jives with my own experience -- one thing that I didn't really take into mind is that since I use subtitles a lot (which aren't transferred on the initial send), after importing a new book I immediately re-connect to fetch those. But if I'm only updating books already on the device (or fetching read statuses), I'm not in a big hurry for the next re-connect.

Also explains why rebooting the device fixed it: it would've made Kobo shut down whatever it was doing.

[NiLuJe]

@davidfor:

Originally Posted by NiLuJe

except for a relevant bit on mx6sll & mx6ull devices, where most of the USBMS machinery is now built-in to the kernel, instead of being pluggable modules (this *will* affect timings, which is a sensible issue on NTX boards ^^).

That's what required the shell script changes, as loading those now fails .

(c.f., this for gnarlier details).