How I got 4000 plus books to show - Page 7

sealbeater · 01-02-2010, 07:23 PM

Quote:

Originally Posted by elcreative

Not quite sure what you're doing but using the last but one set of hacks I have approx 25k of books on an SD card (around 35% full) organised in alpha folders then split to sub-alpha then by author... works like a charm... I can find a book far faster than when I had 6.5k on with standard Sony system... and even moving to IM and getting ready to read, is still faster than find and start on normal system.

Better still, the whole reader responds more quickly without the overhead of managing many files itself... and you can make subject specific cards as no wait for rescan when changing...

Oh yes, I tested with a few books then tried around 2k before going for it and loaded the 25k on a 16GB SDHC card in one go... tooks while to copy as you'd expect but went perfectly...

I now have what I wanted... a library in my pocket where I choose what I want to read when I want to read rather than having to choose a selection beforehand... as someone else said... a library...

Interesting, ok, I'll try again, I am capable of creating a .noscan file for the other poster but I did run into that limit, I will try again, it's good to hear that you have gotten so many, I too, am using a 16GB SDHC, a class 10, when I had a notion of class level possibly causing corruption.

I'll reformat and try again and will let the forum know.

Congratulations by the way, that must be an awesome feeling.

colinsky · 01-08-2010, 01:56 AM

I find this whole thread quite interesting as I (with my PRS-300) have explictly moved in the opposite direction -- I've found the waiting scan upon disconnection/reboot unwieldy enough that I explicity shrunk my on-device library down under 100 books. Everything else sits on the PC ready to be transferred when I need it.

(Incidentally, this probably wouldn't have been such a problem for me if I weren't getting automatic reboots once or twice a week when switching book formats. Hopefully that will be addressed at some point?)

sealbeater · 01-10-2010, 05:51 PM

This is just a quick message, but I redid and still am unable to get a lot of my books accessible. Unfortunate. I got a mesg from someone which seems to confirm an issue with exceeding 4gigs. Anyway, I'll paste a screenshot later.

elcreative · 01-10-2010, 06:25 PM

Would be interested in what you've been told as I've got over 6Gig on my card already... just used my masterset of file/folder layout and dragged onto card with .noscan file in an SDHC enabled card reader. When finished I put card into 505 and turned it on and it worked...

Quote:

Originally Posted by sealbeater

This is just a quick message, but I redid and still am unable to get a lot of my books accessible. Unfortunate. I got a mesg from someone which seems to confirm an issue with exceeding 4gigs. Anyway, I'll paste a screenshot later.

sealbeater · 01-12-2010, 05:43 AM

Make sure you can in fact, access all of your content. I was able to do the same, however, with some folders, when I enter them, there is no content present via the Sony Reader's file manager but the files exists and are readable via computer.

elcreative · 01-12-2010, 03:20 PM

Now all is clear... I dumped all on my card and everything showed up on computer... then I started reading after a quick glance at the folder structure. Reading a series in my SF L folder... no problem. After your message below, I checked further down and at around folder Mi... no contents showing. Once I realised your actual problem, and checked my card, all became clear... cards usually formatted as FAT32 and that gets a bit sticky with anything over 4GB and/or more than 64K directory entries (and that's clusters not files)... now to find out if can use another format for card or maybe partition into 4GB sections or... Meanwhile we're stuck

Quote:

Originally Posted by sealbeater

Make sure you can in fact, access all of your content. I was able to do the same, however, with some folders, when I enter them, there is no content present via the Sony Reader's file manager but the files exists and are readable via computer.

kartu · 01-13-2010, 04:19 AM

Guys, PRS+ (aka "folder hack") now supports disabling external card scan via settings menu (in addition to .noscan).

Sony is NOT mounting external SD/MS drives, but accessing them using some mtools like utility which is bugged (at least when reading non ascii folders/files).

I'd try to format cards using FAT32 but with huge cluster size (I guess 16k is max allowed?) and check if that helps.

Mounting it linux way is still an option to try in the future. Actually tech-savy users could try it right away, using igorsk's flasher's "run script.sh". To mount SD card use:
UPDATE:

Code:

mkdir -p /tmp/sd_card
mount -t vfat /dev/sdmscard/r5c807a1 /tmp/sd_card

You'll have to modify your "Browse Folder" settings (set imRoot variable to corresponding path, e.g. "/tmp/sd_card").

Check if you could see all files this way.

kartu · 01-13-2010, 04:34 AM

Quote:

Originally Posted by sealbeater

Not to beat a dead horse, but I've tried the folders hack with noscan and it still stalls out at around 2 thousand books. If and when the author gets it to the point where it can load the cache.xml file without rescaning the card, this might be the way to go, as you can create the cache.xml file though other means. As it stands now, the "noscan" option acts as a simple file manager but you have to copy the book from SD to internal memory so that's a bit kludgy but it would work if it could actually load thousands of books. The firmware still seems to suffer from the same symptoms but I'm going to load books slow, I've done a bit of experimentation but haven't really had a chance to dig into it, maybe this weekend.

Your right tho, this thread did start a long time ago, I'm glad the author of the folders hack is working on it as well, it would be very nice.

Two things to note here:
1) Reading cache.xml won't help seeing all books. There is a bug ("limitation") in libuss.so.
2) Whether reading cache.xml is feasible at all, is an open question. On the one hand, one "saves" the time to scan the file system. On the other, what should one do with gazillion books in "books by <whatever>"?

It was reported, that when you have loaded about 1300 books (it might depend on books if course) the ebook slows down:
https://www.mobileread.com/forums/sho...&postcount=282

So far first aim would be "hot loading" books from SD card, without rescan. Then adding the "load cache.xml" support (without it, one won't be able to comfortably work with bookmarks etc). So basically it would be "you have thousands or tens of thousands of books on card, but only those that you actually read in cache.xml"

sealbeater · 01-14-2010, 04:21 AM

Quote:

Originally Posted by kartu

Two things to note here:
1) Reading cache.xml won't help seeing all books. There is a bug ("limitation") in libuss.so.
2) Whether reading cache.xml is feasible at all, is an open question. On the one hand, one "saves" the time to scan the file system. On the other, what should one do with gazillion books in "books by <whatever>"?

It was reported, that when you have loaded about 1300 books (it might depend on books if course) the ebook slows down:
https://www.mobileread.com/forums/sho...&postcount=282

So far first aim would be "hot loading" books from SD card, without rescan. Then adding the "load cache.xml" support (without it, one won't be able to comfortably work with bookmarks etc). So basically it would be "you have thousands or tens of thousands of books on card, but only those that you actually read in cache.xml"

1) I don't know what the limitation is, but surely it will help.
2) I'm getting really tired of getting asked this question.

I've been stuffing my reader full of books for a long time, I don't care what has been reported, I've never experienced any type of "slowdown" or "battery life loss" so please let's kill that rumor.

The reason I'm fixated with cache.xml is because all things being equal, if there's no scan of the card, there's no chance of corruption, hence the file manager's dependence on cache.xml for book locations will be unimpeded. I'm completely open to my understanding being flawed, but since there are other ways of generating a good cache file, I thought it was a worthwhile direction.

In a previous post, you mentioned a way of mounting the sd card the linux way. Can you elaborate on how this can be achieved, I saw igorsky's run script.sh method but don't that mean you have to boot off the flash utility? Can it be mounted this way via a change in your code somewhere?

And thanks again for your efforts.

kartu · 01-14-2010, 05:31 AM

sealbeater

Quote:

I've been stuffing my reader full of books for a long time, I don't care what has been reported, I've never experienced any type of "slowdown" or "battery life loss" so please let's kill that rumor.

Well, it's not a rumor. The guy describes circumstances quite clearly.

Quote:

In a previous post, you mentioned a way of mounting the sd card the linux way. Can you elaborate on how this can be achieved, I saw igorsky's run script.sh method but don't that mean you have to boot off the flash utility?

I'm not sure I understand what you mean by "boot off", but:
you put whatever commands you want executed into the script.sh, start igorsk's flasher (by inserting corresponding SD/MS card). In flasher, press 3, it will run script.sh. You can then exit the flasher by pressing the "menu" button.

elcreative · 01-14-2010, 06:21 PM

Sealbeater (& Kartu)

I've now spent some time looking at the number of books and card memory usage availability. Note that I am using PRS+ with card scan disabled... this is to get quantity on the 505 without waiting for a lengthy rescan when adding new books.

Originally I put around 25K books on a 16GB SDHC (using about 6.5GB of the card)... all files and folders accessible (and visible) on computers (both PC and Mac). When installed into the 505 around 16K of files and folders visible and accessible, the rest of the folders showed no content. Checking showed around 16K files folders/4GB usable.

Cleared card and tried with 16GB & 8GB SDHC and 8GB MS... results the same on SDHC cards... at around 16K of files or 4GB of space used, nothing further is visible/accessible on the 505... so far 8GB MS OK but not yet approached 16K/4GB limits.

It looks like a limit of the Sony OS that either 16K of books or 4GB forms a limit for actual visibility/access but by using the SDHC and MS, I now have access to just under 25K of books with scanning disabled. It works fine... browse Files/Folders, click on book and move to internal memory to read. The real annoyance is, that it makes an ability to access 16GB SDHC cards, rather pointless...

As an aside concerning slowdown with lots of books, before using PRS+, I tried various things to see max I could get on a card and, even with 6.5K books on the card, had no slowdown in access/page turn etc... The slowdown came with restarting after adding new books, at this point up to 3 hours to scan and restart which was still usable by doing it overnight but not really comfortable as it would also take that time to restart after a crash... and the machine was definitely less stable with a lot on a scanned memory card (also solved by using PRS+ noscan). The real slowdown I experienced was with very large (several thousand pages) books, in particular PDFs. By the time I got toward the end of one 3K page PDF, page turns were taking 10-15 seconds.

kartu · 01-15-2010, 01:36 AM

elcreative
The limit of 4k files 16Gb books sounds very very strange. How does libuss.so know how many files are out there? Does it read entire allocation table? Hm...

Could it be, that there simply were too many files per folder etc?
How big are clusters on your card?

elcreative · 01-15-2010, 09:24 AM

Kartu,

Thanks for reply, slight misunderstanding... the number of books has been around 16K with a card usage of around 4GB... cluster size not relevant as have used several different cards of 8GB and 16GB. I think the number of files is probably irrelevant and the real limit is using the first 4GB of the card lets you see everything, go over that and everything written after 4GB is not visible in Files/Folders. When I put nearly 7GB on and first noticed the problem (thanks to Sealbeater's comments), I removed more than 3GB of files and still didn't get access to anything extra on the card, visibility failed at the same point but when I wiped all files and reinstalled keeping under 4GB data then everything worked perfectly on all cards.

It looks like the eReader system won't access over the first 4GB of data space in Files/Folders as this seems to have been Sealbeater's experience with a smaller number of (presumably) bigger files.

kartu · 01-16-2010, 10:33 AM

elcreative
By the way, how many folders/files did you have in the root folder?

elcreative · 01-16-2010, 11:15 AM

Originally had 4 folders in root with between 14 and 43 folders at next level. Deepest folder nesting 3 levels and no more than 150 files in a folder... tho majority less than 20 files. I've tried a few more cards and always end at the same point with up to 4GB no problem... anything over not visible from within Files/Folder view. Be good to have a solution but it's a lot more usefule than trying to work with Sony's own system and it's incredibly long scan times once you get over a few thousand files

Quote:

Originally Posted by kartu

elcreative
By the way, how many folders/files did you have in the root folder?

01-13-2010, 04:19 AM	#97
kartu PRS+ author Posts: 1,637 Karma: 2446233 Join Date: Dec 2007 Device: Sony PRS-300, 505, 600, 650, 950	Guys, PRS+ (aka "folder hack") now supports disabling external card scan via settings menu (in addition to .noscan). Sony is NOT mounting external SD/MS drives, but accessing them using some mtools like utility which is bugged (at least when reading non ascii folders/files). I'd try to format cards using FAT32 but with huge cluster size (I guess 16k is max allowed?) and check if that helps. Mounting it linux way is still an option to try in the future. Actually tech-savy users could try it right away, using igorsk's flasher's "run script.sh". To mount SD card use: UPDATE: Code: mkdir -p /tmp/sd_card mount -t vfat /dev/sdmscard/r5c807a1 /tmp/sd_card You'll have to modify your "Browse Folder" settings (set imRoot variable to corresponding path, e.g. "/tmp/sd_card"). Check if you could see all files this way. Last edited by kartu; 01-13-2010 at 04:23 AM.

01-15-2010, 09:24 AM	#103
elcreative Wizard Posts: 2,888 Karma: 5875940 Join Date: Dec 2007 Device: PRS505, 600, 350, 650, Nexus 7, Note III, iPad 4 etc	Kartu, Thanks for reply, slight misunderstanding... the number of books has been around 16K with a card usage of around 4GB... cluster size not relevant as have used several different cards of 8GB and 16GB. I think the number of files is probably irrelevant and the real limit is using the first 4GB of the card lets you see everything, go over that and everything written after 4GB is not visible in Files/Folders. When I put nearly 7GB on and first noticed the problem (thanks to Sealbeater's comments), I removed more than 3GB of files and still didn't get access to anything extra on the card, visibility failed at the same point but when I wiped all files and reinstalled keeping under 4GB data then everything worked perfectly on all cards. It looks like the eReader system won't access over the first 4GB of data space in Files/Folders as this seems to have been Sealbeater's experience with a smaller number of (presumably) bigger files. Last edited by elcreative; 01-15-2010 at 09:45 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Books show in italics	dcpyke	Sony Reader	6	11-20-2009 11:49 AM
When 150 becomes 4000!	Ads	Sony Reader	7	09-03-2009 11:22 AM
Help - My Books now show up in reverse order...	muñeca	Fictionwise eBookwise	3	03-12-2008 08:48 PM

01-08-2010, 01:56 AM	#92
colinsky Addict Posts: 240 Karma: 3500000 Join Date: Sep 2009 Device: Sony PRS-300, PRS-T1, PRS-T3	I find this whole thread quite interesting as I (with my PRS-300) have explictly moved in the opposite direction -- I've found the waiting scan upon disconnection/reboot unwieldy enough that I explicity shrunk my on-device library down under 100 books. Everything else sits on the PC ready to be transferred when I need it. (Incidentally, this probably wouldn't have been such a problem for me if I weren't getting automatic reboots once or twice a week when switching book formats. Hopefully that will be addressed at some point?)

01-10-2010, 05:51 PM	#93
sealbeater Banned Posts: 666 Karma: 1752814 Join Date: Jan 2008 Device: Sony Reader PRS-505 : Onyx Boox Max : Sony PRS-900 : Onyx Kepler Pro	This is just a quick message, but I redid and still am unable to get a lot of my books accessible. Unfortunate. I got a mesg from someone which seems to confirm an issue with exceeding 4gigs. Anyway, I'll paste a screenshot later.

01-12-2010, 05:43 AM	#95
sealbeater Banned Posts: 666 Karma: 1752814 Join Date: Jan 2008 Device: Sony Reader PRS-505 : Onyx Boox Max : Sony PRS-900 : Onyx Kepler Pro	Make sure you can in fact, access all of your content. I was able to do the same, however, with some folders, when I enter them, there is no content present via the Sony Reader's file manager but the files exists and are readable via computer.

01-14-2010, 06:21 PM	#101
elcreative Wizard Posts: 2,888 Karma: 5875940 Join Date: Dec 2007 Device: PRS505, 600, 350, 650, Nexus 7, Note III, iPad 4 etc	Sealbeater (& Kartu) I've now spent some time looking at the number of books and card memory usage availability. Note that I am using PRS+ with card scan disabled... this is to get quantity on the 505 without waiting for a lengthy rescan when adding new books. Originally I put around 25K books on a 16GB SDHC (using about 6.5GB of the card)... all files and folders accessible (and visible) on computers (both PC and Mac). When installed into the 505 around 16K of files and folders visible and accessible, the rest of the folders showed no content. Checking showed around 16K files folders/4GB usable. Cleared card and tried with 16GB & 8GB SDHC and 8GB MS... results the same on SDHC cards... at around 16K of files or 4GB of space used, nothing further is visible/accessible on the 505... so far 8GB MS OK but not yet approached 16K/4GB limits. It looks like a limit of the Sony OS that either 16K of books or 4GB forms a limit for actual visibility/access but by using the SDHC and MS, I now have access to just under 25K of books with scanning disabled. It works fine... browse Files/Folders, click on book and move to internal memory to read. The real annoyance is, that it makes an ability to access 16GB SDHC cards, rather pointless... As an aside concerning slowdown with lots of books, before using PRS+, I tried various things to see max I could get on a card and, even with 6.5K books on the card, had no slowdown in access/page turn etc... The slowdown came with restarting after adding new books, at this point up to 3 hours to scan and restart which was still usable by doing it overnight but not really comfortable as it would also take that time to restart after a crash... and the machine was definitely less stable with a lot on a scanned memory card (also solved by using PRS+ noscan). The real slowdown I experienced was with very large (several thousand pages) books, in particular PDFs. By the time I got toward the end of one 3K page PDF, page turns were taking 10-15 seconds.

01-15-2010, 01:36 AM	#102
kartu PRS+ author Posts: 1,637 Karma: 2446233 Join Date: Dec 2007 Device: Sony PRS-300, 505, 600, 650, 950	elcreative The limit of 4k files 16Gb books sounds very very strange. How does libuss.so know how many files are out there? Does it read entire allocation table? Hm... Could it be, that there simply were too many files per folder etc? How big are clusters on your card?

01-16-2010, 10:33 AM	#104
kartu PRS+ author Posts: 1,637 Karma: 2446233 Join Date: Dec 2007 Device: Sony PRS-300, 505, 600, 650, 950	elcreative By the way, how many folders/files did you have in the root folder?