Hacking SD card file system

PeterT · 05-21-2013, 09:11 AM

After some discussions got started over in the Upgrade thread on changing the SD card to a different format than the default FAT32 I thought I would move the discussion over here.

Original request:

Quote:

Originally Posted by jlynton

My request is for it to be possible to use an ext3 (or ext4) formatted microSD card, mainly to get around the file limitations in the images directory.

My initial reply:

Quote:

Originally Posted by PeterT

Maybe one of the tweakers in the Kobo Developer's Corner could look into this. I have a gut feeling that what is needed is a change to the script used for mounting the SD card.

Quote:

Originally Posted by \usr\local\Kobo\uDev\sd

Code:

DEV="/dev/mmcblk1p1"

if [ "$DEV" == "$DEVNAME" ]; then
	if [ "$ACTION" == "add" ]; then
		dosfsck -a -w $DEVNAME
		mount -r -t vfat -o shortname=mixed $DEVNAME /mnt/sd
		if [ $? != 0 ]; then
#			mount -r $DEVNAME /mnt/sd
#			if [ $? != 0 ]; then
				echo sd mount fail $DEVNAME >> /tmp/nickel-hardware-status &
#			else
#				echo sd $ACTION $DEVNAME >> /tmp/nickel-hardware-status &
#			fi
		else
			echo sd $ACTION $DEVNAME >> /tmp/nickel-hardware-status &
		fi
	fi
fi

It looks like the ability to mount non FAT32 formats was there at one point, and has now been commented out.

More discussion:

Quote:

Originally Posted by tshering

If one makes changes there, one might also consider adjusting usr/local/Kobo/udev/usb.

Quote:

Originally Posted by jlynton

This looks interesting, is there an easy way to access these files (glo, without removing internal microSD card)?

Quote:

Originally Posted by tshering

One way of modifying these files is by using the upgrade mechanism. Unpack the file "KoboRoot.tgz" from the FW update file, which you can find here ,and then unpack its (KoboRoot.tgz's) content. After modifying some files you can pack them again into "KoboRoot.tgz" (you have of course to preserve the path of each modified file, but there is no need to include all the other files). Copy then "KoboRoot.tgz" to the folder ".kobo" of the reader. After unplugging the reader, it will copy the content of "KoboRoot.tgz" to the system partition. Confer the paragraph "The Kobo upgrade mechanism" of "Hacking the Kobo Touch for DUMMIES". This part is valid for Glo too.
Word of caution: Preserve lower and upper case of file and path names (e.g. udev, not uDev).
If you mess things up, you might have to factory reset the reader. If you have a lot of annotations and highlights, you might consider making a back up of .kobo/KoboReader.sqlite.

Quote:

Originally Posted by jlynton

Thanks, this looks like something I could try on my day off. I have looked at the two files usr/local/Kobo/udev/usb and usr/local/Kobo/udev/sd, both have the commented out section that PeterT refers to. I assume that one refers to the internal microSD cards FAT32 partition, and the other to the external microSD card, do you know which is which?

Quote:

Originally Posted by PeterT

Actually I believe that the sd is used only for the mounting of the external sd card; note that it mounts the card on /mnt/sd. usb is used for when the device is attached to your PC.

/mnt/onboard appears to be used for the partition that is accessible to us normally, and /mnt/sd to the external SD card.

I think that this section

Quote:

Originally Posted by /usr/local/Kobo/udev/usb

Code:

	if [ -e $DEV ]; then
		mount -r -t vfat -o shortname=mixed $DEV /mnt/sd
#		if [ $? != 0 ]; then
#			mount -r $DEV /mnt/sd
#		fi
	fi

should be similar to the code in /usr/local/Kobo/udev/sd

jlynton · 05-21-2013, 11:26 AM

While I am a seasoned Linux user, I am merely a toddler when it comes to coding. Would I be right to assume that if I removed the comments from

Code:

#			mount -r $DEVNAME /mnt/sd
#			if [ $? != 0 ]; then
				echo sd mount fail $DEVNAME >> /tmp/nickel-hardware-status &
#			else
#				echo sd $ACTION $DEVNAME >> /tmp/nickel-hardware-status &
#			fi

Then I should be able to mount an ext3 formatted microSD card?

PeterT · 05-21-2013, 02:22 PM

I believe so; the advice would be to make the changes in both the sd and usb scripts. Possibly some "smartness" could be added by seeing if the file -s command on the raw device works; from my reading file -s $DEVNAME should identify the file system type, and would allow selective running of the correct fsck command.

jlynton · 05-22-2013, 02:09 AM

If I understand correctly, you are saying that the line should read

Code:

mount -r -s $DEVNAME /mnt/sd

is that correct?

davidfor · 05-22-2013, 03:13 AM

I don't think that is what Peter means.

From the code, it is attempting to mount the SD card as FAT32. If that fails, it gives an error. The commented out lines attempt to mount the SD card without specifying a file system type. I think Peter is suggesting running a command to test the file system and from that deciding the parameters for the mount command.

There is a good chance that simply uncommenting the lines will work. The file system type is optional for the mount command as it can determine this if the appropriate file systems are available to it. I would expect a SD card formatted with ext3 or ext4 to work. But NTFS or HFS probably wouldn't. Or, as you know what file system you formatted the card with, you can add the appropriate parameter to second mount command.

In any case, the test won't damage anything so long as you are prepared to do a factory reset to get the device working again.

jlynton · 05-22-2013, 04:40 AM

Thank you both, David and Peter, for your help and suggestions.

I will try to uncomment the lines on both files (usr/local/Kobo/udev/usb and usr/local/Kobo/udev/sd), and use an ext3 formatted microSD sometime this weekend. I will then report back as to how it goes.

PeterT · 05-22-2013, 09:58 AM

One thiong I'm wondering about, and unfortunately, don't have enough knowledge to answer.

If the SD card IS in ext2 / ext3 / ext4 format, will it be accessible via the PC when connected via USB? I've got a gut feeling that the answer is NO.

If that IS the case it kind of makes the whole exercise moot.

davidfor · 05-22-2013, 10:11 AM

It will depend on the OS running on the PC. A Linux system should have no problem. A Windows system won't read it. But, there are installable ext3/ext4 file systems available for Windows. I haven't tried one for a while. I don't know what a Mac would do, but my guess would be either native support or easily added.

jgoguen · 05-22-2013, 12:37 PM

For OS X, the only option I'm aware of is installing OSXFUSE and its ext2 module. For Windows, ext2fsd but I think that defaults to read-only support and doesn't support ext3/4 journaling.

PeterT · 05-22-2013, 12:37 PM

So that means that while it is likely possible that we can make the external SD card into a different file system type that for most users it will not be worthwhile; Windows will not recognize it which means neither Kobo desktop or calibre will either.

jlynton · 05-22-2013, 12:49 PM

Ok, I have bitten the bullet (and missed my siesta), and can report a complete success!

To clarify, I do use Linux on my desktop (and everywhere else!). I did as I said I would...

Quote:

Originally Posted by jlynton

I will try to uncomment the lines on both files (usr/local/Kobo/udev/usb and usr/local/Kobo/udev/sd), and use an ext3 formatted microSD sometime this weekend. I will then report back as to how it goes.

The only (small) hiccup was that I had to change the microSD card's permissions to make it read/write to me before calibre would see the card. Otherwise there were no problems, and as far as I can tell everything is working perfectly :-)

jgoguen · 05-22-2013, 01:58 PM

Wouldn't you also need the filesystem to be world-writable? So that the cover images can be written and the database updated? Or does the Kobo software run as root?

GeoffR · 05-22-2013, 02:17 PM

Nice work, it is good to know this option is available if Kobo don't come up with a better way to store cover images by the time I reach the FAT limit.

I don't use Windows so haven't tried it, but according to Wikipedia Ex2IFS will allow mounting a read/write ext2/3 filesystem in Windows.

Edit: Journaling support is not necessary anyway, FAT doesn't have a journal so you don't lose anything by formatting the card ext2 without a journal.

jlynton · 05-23-2013, 08:17 AM

Quote:

Originally Posted by jgoguen

Wouldn't you also need the filesystem to be world-writable? So that the cover images can be written and the database updated? Or does the Kobo software run as root?

The kobo sw appears to run as root.

jlynton · 05-23-2013, 08:23 AM

Quote:

Originally Posted by GeoffR

Nice work, it is good to know this option is available if Kobo don't come up with a better way to store cover images by the time I reach the FAT limit.

I don't use Windows so haven't tried it, but according to Wikipedia Ex2IFS will allow mounting a read/write ext2/3 filesystem in Windows.

Edit: Journaling support is not necessary anyway, FAT doesn't have a journal so you don't lose anything by formatting the card ext2 without a journal.

Yes, ext2 would be fine as well.

Everything is still working beautifully, with the added bonus that calibre to glo transfers seem to be about 2X faster, and even the browsing of shelves etc. withing the glo seems a bit snappier.

EXT2IFS would be a possibility for windows users, but a real solution would be to ditch M$ and use Linux instead ;-)

05-21-2013, 11:26 AM	#2
jlynton Evangelist Posts: 414 Karma: 216544 Join Date: Feb 2012 Device: Kobo Clara HD, Moaan Inkpalm 5.	While I am a seasoned Linux user, I am merely a toddler when it comes to coding. Would I be right to assume that if I removed the comments from Code: # mount -r $DEVNAME /mnt/sd # if [ $? != 0 ]; then echo sd mount fail $DEVNAME >> /tmp/nickel-hardware-status & # else # echo sd $ACTION $DEVNAME >> /tmp/nickel-hardware-status & # fi Then I should be able to mount an ext3 formatted microSD card?

05-22-2013, 02:09 AM	#4
jlynton Evangelist Posts: 414 Karma: 216544 Join Date: Feb 2012 Device: Kobo Clara HD, Moaan Inkpalm 5.	If I understand correctly, you are saying that the line should read Code: mount -r -s $DEVNAME /mnt/sd is that correct?

05-22-2013, 02:17 PM	#13
GeoffR Wizard Posts: 3,821 Karma: 19162882 Join Date: Nov 2012 Location: Te Riu-a-Māui Device: Kobo Glo	Nice work, it is good to know this option is available if Kobo don't come up with a better way to store cover images by the time I reach the FAT limit. I don't use Windows so haven't tried it, but according to Wikipedia Ex2IFS will allow mounting a read/write ext2/3 filesystem in Windows. Edit: Journaling support is not necessary anyway, FAT doesn't have a journal so you don't lose anything by formatting the card ext2 without a journal. Last edited by GeoffR; 05-22-2013 at 02:28 PM. Reason: fix url

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
kindle 3 hardware hacking (micro usb & sd card)	damaru	Amazon Kindle	5	03-16-2012 08:19 AM
360 Pocketbook 360 and SD card file system error	readerreader	PocketBook	2	08-29-2011 12:10 AM
K2 system file access	dwallbaum	Amazon Kindle	2	09-08-2010 04:32 PM
File System Information	Adam B.	iRex	0	10-29-2008 08:46 AM
how to clean more disk space in root file system to upgrade system	chinaet	iRex	1	12-18-2006 03:54 PM

05-21-2013, 02:22 PM	#3
PeterT Grand Sorcerer Posts: 12,160 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	I believe so; the advice would be to make the changes in both the sd and usb scripts. Possibly some "smartness" could be added by seeing if the file -s command on the raw device works; from my reading file -s $DEVNAME should identify the file system type, and would allow selective running of the correct fsck command.

05-22-2013, 03:13 AM	#5
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	I don't think that is what Peter means. From the code, it is attempting to mount the SD card as FAT32. If that fails, it gives an error. The commented out lines attempt to mount the SD card without specifying a file system type. I think Peter is suggesting running a command to test the file system and from that deciding the parameters for the mount command. There is a good chance that simply uncommenting the lines will work. The file system type is optional for the mount command as it can determine this if the appropriate file systems are available to it. I would expect a SD card formatted with ext3 or ext4 to work. But NTFS or HFS probably wouldn't. Or, as you know what file system you formatted the card with, you can add the appropriate parameter to second mount command. In any case, the test won't damage anything so long as you are prepared to do a factory reset to get the device working again.

05-22-2013, 04:40 AM	#6
jlynton Evangelist Posts: 414 Karma: 216544 Join Date: Feb 2012 Device: Kobo Clara HD, Moaan Inkpalm 5.	Thank you both, David and Peter, for your help and suggestions. I will try to uncomment the lines on both files (usr/local/Kobo/udev/usb and usr/local/Kobo/udev/sd), and use an ext3 formatted microSD sometime this weekend. I will then report back as to how it goes.

05-22-2013, 09:58 AM	#7
PeterT Grand Sorcerer Posts: 12,160 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	One thiong I'm wondering about, and unfortunately, don't have enough knowledge to answer. If the SD card IS in ext2 / ext3 / ext4 format, will it be accessible via the PC when connected via USB? I've got a gut feeling that the answer is NO. If that IS the case it kind of makes the whole exercise moot.

05-22-2013, 10:11 AM	#8
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	It will depend on the OS running on the PC. A Linux system should have no problem. A Windows system won't read it. But, there are installable ext3/ext4 file systems available for Windows. I haven't tried one for a while. I don't know what a Mac would do, but my guess would be either native support or easily added.

05-22-2013, 12:37 PM	#9
jgoguen Generally Awesome Person Posts: 1,061 Karma: 2178845 Join Date: Jan 2013 Location: /dev/kmem Device: Kobo Clara HD, Kindle Oasis	For OS X, the only option I'm aware of is installing OSXFUSE and its ext2 module. For Windows, ext2fsd but I think that defaults to read-only support and doesn't support ext3/4 journaling.

05-22-2013, 01:58 PM	#12
jgoguen Generally Awesome Person Posts: 1,061 Karma: 2178845 Join Date: Jan 2013 Location: /dev/kmem Device: Kobo Clara HD, Kindle Oasis	Wouldn't you also need the filesystem to be world-writable? So that the cover images can be written and the database updated? Or does the Kobo software run as root?