![]() |
#1 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 90
Karma: 1059
Join Date: Sep 2011
Location: Canada
Device: OnyxBook M96, enTourage Pocket eDGe (retired)
|
Rebuilding the filesystem to increase app storage
Hi Everyone!
I have managed to hack a new UBI image (based on the Dingo golden update one) with a larger space for applications (taken from the storage volume, since that can go into the SD card). This is *NOT* user-friendly, and I am not sure it works properly. I have found the following issues after flashing with the new image:
On the other hand, if things go bad the device should be recoverable using the Dingo golden update (I lost count of the number of times I have flashed that ![]() In order to make run the script that makes the new image you need:
Other technical notes:
The script I have used follows. Again, *DO NOT USE* unless you know exactly what you are doing! Code:
#! /bin/sh # This script grabs the UBI image from the Pocket Edge Dingo golden update and # creates a modified UBI image where the "userdata" (where apps are installed) # and the "storage" (where the library resides) can be custom-sized (with some # restraints). # # Requires: # * A recent enough kernel with nandsim and ubifs. # * mtd-utils (probably a recent version) # * sed # * The android_ubi.img.crc file from the Dingo Golden update. # * The update.sh file from the Dingo Golden update. # * It needs to be run as root. # # Creates new-android_ubi.img.crc and new-update.sh. Put these two files into # the original update.zip and update the system in your Pocket Edge. # New size for the storage volume and userdata volume in logical eraseblocks # (LEBs); each LEB is 504Kb long. The sum of these must be equal or less to # 6357 (if it's less it will work, but you will be wasting space). # NOTE: Keep in mind that the actual usable space will be at least ~70 Mib # smaller because of filesystem structures. STO_SIZE=1200 # ~590 MiB USD_SIZE=5157 # ~2.5 GiB # Temporary directory. BE CAREFUL, it will be wiped out and created each time! TMPDIR=/tmp/muckrom # It should not be necessary to edit anything below this. # File names. CRC_IMAGE=android_ubi.img.crc IMAGE=android_ubi.img # Check for an image without CRC. If there is none, and the CRC image is # present, remove it. if [ ! -r $IMAGE ] then echo "Eliminating the CRC from the image file $CRC_IMAGE." dd if=$CRC_IMAGE of=$IMAGE bs=512k count=327 fi # First, load up the NAND simulator, trying to reproduce the layout in the PE. # The first partition will be unused, and the second one is the one with the # UBI volumes (the total NAND size is 8192 LEBs, and the second partition size # is 7864 LEBs. echo "Loading the NAND simulator." modprobe ubifs modprobe nandsim first_id_byte=0xec second_id_byte=0xd7 third_id_byte=0x00 fourth_id_byte=0x36 parts=328,7864 cache_file=/tmp/nand.img # This will be the MTD device to use. MDEV=/dev/mtd1 # Copy (bit-for-bit) the image into the simulated NAND partition. echo "Uploading image into simulated NAND." dd if=$IMAGE of=$MDEV bs=512k count=327 # Verify the copy, to make sure nothing went wrong. echo "Verifying the image copied to the simulated NAND." nanddump -o -b -l `echo 327*512*1024 | bc` -s 0 -f verify.img $MDEV cmp $IMAGE verify.img if [ $? -ne 0 ] then echo "Problem copying the image to the simulated NAND!" rm verify.img modprobe -r nandsim rm -f /tmp/nand.img exit 1 fi echo "Dump verified OK!" rm verify.img # Attach the UBI volumes (force UBI not to use subpages, for some reason the # NAND simulator reports 1Kb subpagesm while the PE hardware does not seem to # support them). echo "Attaching UBI device." ubiattach -p $MDEV -O 4096 # Create the temporary directory. echo "Creating the temporary directory $TMPDIR." rm -r $TMPDIR mkdir -p $TMPDIR # Create the directories and mount the UBI images. echo "Mounting original UBI images." for vol in system userdata cache storage do mkdir $TMPDIR/$vol mount -t ubifs ubi:$vol $TMPDIR/$vol done # Make the UBI filesystem images. echo "Making the new UBI filesystem images..." echo " ...system" mkfs.ubifs -v -r $TMPDIR/system -m 4096 -e 516096 -c 712 -j 10584KiB -R 0 -x favor_lzo -X 1 new-system.img echo " ...userdata" mkfs.ubifs -v -r $TMPDIR/userdata -m 4096 -e 516096 -c $USD_SIZE -j 10584KiB -R 0 -x favor_lzo -X 1 new-userdata.img echo " ...cache" mkfs.ubifs -v -r $TMPDIR/cache -m 4096 -e 516096 -c 712 -j 17640KiB -R 4536KiB -x favor_lzo -X 1 new-cache.img echo " ...storage" mkfs.ubifs -v -r $TMPDIR/storage -m 4096 -e 516096 -c $STO_SIZE -j 32760KiB -R 4536KiB -x favor_lzo -X 1 new-storage.img # Unmount the UBI images. echo "Unmounting original UBI images." for vol in system userdata cache storage do umount $TMPDIR/$vol done # Then, let's detach UBI from the device. echo "Detaching UBI device." ubidetach -p $MDEV # We can remove the NAND simulator now. echo "Removing the NAND simulator." modprobe -r nandsim rm -f /tmp/nand.img modprobe -r ubifs # Make the big, ubinized, image. cat <<EOF > new-ubinize.cfg [system] mode=ubi image=./new-system.img vol_size=358848KiB vol_id=0 vol_type=dynamic vol_name=system [userdata] mode=ubi image=./new-userdata.img vol_size=`echo 504*$USD_SIZE | bc`KiB vol_id=1 vol_type=dynamic vol_name=userdata [cache] mode=ubi image=./new-cache.img vol_size=358848KiB vol_id=2 vol_type=dynamic vol_name=cache [storage] mode=ubi image=./new-storage.img vol_size=`echo 504*$STO_SIZE | bc`KiB vol_id=3 vol_type=dynamic vol_name=storage EOF echo "Running ubinize." ubinize -o new-$IMAGE -p 512KiB -m 4096 new-ubinize.cfg # Add the CRC to the end. # After this, the image should be ready to put in a custom "update.zip" file. echo "Adding CRC to the modified image." CRC=`/usr/sbin/ubicrc32 new-$IMAGE` perl -e "print pack L, ${CRC}" > crc_file.dat cat new-$IMAGE crc_file.dat > new-$IMAGE.crc rm -f crc_file.dat # NOTE: The custom "update.sh" should have VFAT_NUM_BLOCKS equal to # $STO_SIZE*504 (the new number of blocks in the storage filesystem) # To that, we substract the journal and root reserved space to make sure the # image will never be larger than the available space. NEW_BLOCKS=`echo $STO_SIZE*504-32760-4536 | bc` sed s/VFAT_NUM_BLOCKS=2617152/VFAT_NUM_BLOCKS=${NEW_BLOCKS}/ update.sh > new-update.sh |
![]() |
![]() |
![]() |
#2 |
Guru
![]() ![]() ![]() ![]() ![]() Posts: 733
Karma: 443
Join Date: May 2011
Location: UK
Device: Pocket Edgex2(1unborked), Alex, Nook B&W, Nook Color, Nook STR
|
while this is good, why dont u provide your golden modified as a file?
|
![]() |
![]() |
Advert | |
|
![]() |
#3 | |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 90
Karma: 1059
Join Date: Sep 2011
Location: Canada
Device: OnyxBook M96, enTourage Pocket eDGe (retired)
|
Quote:
A more trivial reason is that I don't know what would be an ideal split between app space and storage. With the caveats above, plus the fact that this will *ERASE ALL DATA AND APPS*, here it goes: 1) Downgrade all the way to Dingo (you might need to use one of the hellbent updates for that). 2) Download the file from http://www.megaupload.com/?d=93P0FWVB. 3) Save the file as "update.zip" in the root directory of a USD drive or a MicroSD card (I had more luck with the microSD card, but YMMV). 4) Flash the PE with it. If everything goes right, it will reboot. If there is a problem, the PE will power off instead of rebooting. 4b) If something went wrong, there should be an "upd.log" file in the root directory of the medium used for the upgrade. That might give some info on what happened. 5) Upgrade to Ermine (don't bother with Dingo, it doesn't work well for some reason). 5b) You can upgrade to Allmine, if you want (I did it and it seems to work). 6) You'll probably want to move the library to SD. I don't know if that works yet because I haven't received my new card. 7) Fill it up with apps and let me know if it works! 8) Don't complain if your PE sprouts wings and flies away. ![]() Ok, that's it. Good luck everyone! P.S.: It seems to be working on mine, so the obsession-fueled adrenaline rush is over. Now I need to go work. ![]() Last edited by pablob; 11-21-2011 at 10:15 PM. Reason: Added link to update and instructions. |
|
![]() |
![]() |
![]() |
#4 |
Armed with a smile :)
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,421
Karma: 2463560
Join Date: Sep 2009
Location: California, USA
Device: enTourage eDGe & Pocket eDGe, Samsung Galaxy Note II
|
Very cool work you are doing, pablob! Nothing I will attempt, of course, but I'm really impressed that you can figure this out!
![]() |
![]() |
![]() |
![]() |
#5 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 90
Karma: 1059
Join Date: Sep 2011
Location: Canada
Device: OnyxBook M96, enTourage Pocket eDGe (retired)
|
Thanks Filark!
It was a nice learning experience about NAND flash, Linux MTDs and UBI. ![]() Now I'm thinking it should be possible to actually coalesce all the updates (Dingo, Ermine, Allmine) into one single update file (a ROM, you could say)... The question is whether I'll have time to think about that... ![]() |
![]() |
![]() |
Advert | |
|
![]() |
#6 |
Guru
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 614
Karma: 73700
Join Date: Oct 2009
Location: WA, USA
Device: Android, Kindle Paperwhite, lots of ancient readers
|
Excellent work. I'm personally more interested in "re-partitioning" the device so we can add a real android style recovery partition, but this is a wonderful step in the right direction. For that I'd need to shrink one of the images and create a new one. Looks like I could learn quite a bit from this, will have to dig into it tomorrow.
|
![]() |
![]() |
![]() |
#7 |
Guru
![]() ![]() ![]() ![]() ![]() Posts: 733
Karma: 443
Join Date: May 2011
Location: UK
Device: Pocket Edgex2(1unborked), Alex, Nook B&W, Nook Color, Nook STR
|
hip hip hooray. yes. android recovery and larger apps partition is the way forward. really not interested in the internal sd space for anything else as we do have 16 gb + sd cards now. can have the library on different ext sd and no issue on removing the cards as with current method for extending app partition.
|
![]() |
![]() |
![]() |
#8 |
Guru
![]() ![]() ![]() ![]() ![]() Posts: 733
Karma: 443
Join Date: May 2011
Location: UK
Device: Pocket Edgex2(1unborked), Alex, Nook B&W, Nook Color, Nook STR
|
can we dream for a gingerbread rom?
|
![]() |
![]() |
![]() |
#9 |
Gadget Freak
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,169
Karma: 1043832
Join Date: Nov 2007
Location: US
Device: EE, Note 8
|
|
![]() |
![]() |
![]() |
#10 |
Dog Dentist
![]() Posts: 108
Karma: 18
Join Date: Sep 2011
Device: PE, GT540, VM670, GOGHVMU, Qualcomm 1600
|
quite possibly... a lot of gingerbread "rom"s I've seen for phones, basically keep the old kernel (mostly b/c binary blobs for certain device drivers, as we do have on the PE with eink) from the phone's existing rom, and then add as much of gingerbread framework as they can (some of which is kernel dependent, so bits get left out)
|
![]() |
![]() |
![]() |
#11 |
Groupie
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 198
Karma: 1118
Join Date: Jul 2011
Device: Pocket Edge
|
This is fantastic! I was thinking about how to do this myself after I learned how it works from fixing my frozen PE. I was thinking about replacing the existing storage partition and using that space for the data partition since I was using the link2sd method to put stuff there.
This is awesome. ![]() |
![]() |
![]() |
![]() |
#12 | |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 90
Karma: 1059
Join Date: Sep 2011
Location: Canada
Device: OnyxBook M96, enTourage Pocket eDGe (retired)
|
Ideas for recovery...
Quote:
![]() I have been thinking on ways to have recovery working, and this is what I have come up so far:
|
|
![]() |
![]() |
![]() |
#13 |
Guru
![]() ![]() ![]() ![]() ![]() Posts: 733
Karma: 443
Join Date: May 2011
Location: UK
Device: Pocket Edgex2(1unborked), Alex, Nook B&W, Nook Color, Nook STR
|
allmine works fine, at least for me.
An allmine recovery and option to update to any .zip file is all we need, really. menu+rotate should bring up that menu. than no need for serial and tftps from linux. we will have a failsafe recovery and everybody could experiment safely. Agreed on no internal storage, or minimal storage and plenty of space for apps. external storage is enough really if you have a large 8/16/32 gb sd. |
![]() |
![]() |
![]() |
#14 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 90
Karma: 1059
Join Date: Sep 2011
Location: Canada
Device: OnyxBook M96, enTourage Pocket eDGe (retired)
|
Interesting... Yesterday I turned off my PE with no problems. Today, when I turn it on, it goes all the way to the 4 dots and then it stays there, flashing from time to time. It seems the boot up got stuck somewhere, but unfortunately I have no way of knowing where (I don't have the serial port open yet). It might be random, it might be a problem with the filesystem change, who knows...
![]() I wonder if anybody has any idea on how to get a console with the boot messages. That might require a bit of "kernel magic"... |
![]() |
![]() |
![]() |
#15 |
Guru
![]() ![]() ![]() ![]() ![]() Posts: 733
Karma: 443
Join Date: May 2011
Location: UK
Device: Pocket Edgex2(1unborked), Alex, Nook B&W, Nook Color, Nook STR
|
I think you may have bad sectors. Repartitioning could have brought those forward.
|
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Rebuilding library from device | esuli | Library Management | 7 | 11-16-2011 03:33 AM |
Free (Kindle/Sony) I Surrender All: Rebuilding a Marriage Broken by Pornography | arcadata | Deals and Resources (No Self-Promotion or Affiliate Links) | 8 | 10-12-2011 11:56 AM |
missing 'Storage' app? | dwatransit | enTourage eDGe | 6 | 07-28-2011 05:06 PM |
Android APK Rebuilding | techiem2 | enTourage Archive | 7 | 05-25-2010 10:39 PM |
iPhone Two new iPhone Storage Solutions: Zoomit (SD Card Reader) & Boxnet (Cloud Storage) | kjk | Apple Devices | 0 | 02-09-2010 06:20 PM |