02-03-2020, 11:16 PM | #16 | |
Connoisseur
Posts: 57
Karma: 43710
Join Date: Jun 2008
Device: zaurus->palm->iPad->Sony PRS-T1,T2,T3->Kobo Forma&Likebook Ares
|
Quote:
Now, while it is possible to use the file search instead of the file manager, one thing that can not be achieved here is adding a book to favorites. For me, that would speed up things even more, since I could mark books as favorites and access them quickly. However, as far as I can see, the only way to add to favorites is a long press on the entry in the file manager. I wish there was a menu entry or assignable command "Add to favorites" somewhere... All the best, Ctop |
|
02-04-2020, 01:07 AM | #17 |
BLAM!
Posts: 13,477
Karma: 26012492
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
|
Hmm. Define "blazingly fast" ^^.
And, just to double-check, we *are* talking about the "Find a file" menu entry, right? FWIW, on an H2O (whose storage speed sucks ass) w/ ~500 boks (using a classic LN, FN/ single-level hierarchy, i.e., 35 pages), a file search (starting from the Home dir) takes between 6 and 10s, and a Calibre search is instantaneous (and more accurate ;p). |
Advert | |
|
02-04-2020, 06:45 AM | #18 | |
Connoisseur
Posts: 57
Karma: 43710
Join Date: Jun 2008
Device: zaurus->palm->iPad->Sony PRS-T1,T2,T3->Kobo Forma&Likebook Ares
|
Quote:
All the best, Ctop |
|
02-04-2020, 09:47 AM | #19 | |
cosiñeiro
Posts: 1,271
Karma: 2189337
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Quote:
I'm aware that likebooks are really broken in many aspects: - Its launcher have a bunch of memory leaks, which is specially bad on a home launcher. - Its media scanner is pretty broken and won't pause during mtp transfer. - Its media scanner works per folder, not per document. If it crash it will need to start again in the same folder. It crashes a lot. But the forma should be pretty responsive, in spite of vfat. My suggestion would be to focus on the forma: remove the koreader installation (on .adds/koreader, keep a backup somewhere else if you like) and start from scratch. Do not attempt to index anything and report back timmings. |
|
02-04-2020, 12:55 PM | #20 |
Wizard
Posts: 1,613
Karma: 724945
Join Date: Oct 2014
Location: Antwerp
Device: Kobo Aura H2O
|
That sounds excruciatingly slow. Atm I have "602 items, totaling 1.2 GiB" in my biggest folder and that takes less than a second to search through. It's not a flat file structure, but I would think flat should be faster if anything.
Last edited by Frenzie; 02-04-2020 at 12:57 PM. |
Advert | |
|
02-04-2020, 02:03 PM | #21 |
BLAM!
Posts: 13,477
Karma: 26012492
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
|
@Frenzie: I wouldn't be surprised if a flat structure was faster, actually .
I also have a bunch of extremely nasty trees (Hi, Python!) in various hidden dot folders, which we might be traversing? That might explain the discrepancy. EDIT: Yup, probably. It should probably honor our "show hidden folders" setting to prevent that. @pazos/@ctop: Yep, the Forma has *much* better storage. Last edited by NiLuJe; 02-04-2020 at 02:07 PM. |
02-04-2020, 03:22 PM | #22 |
Wizard
Posts: 1,613
Karma: 724945
Join Date: Oct 2014
Location: Antwerp
Device: Kobo Aura H2O
|
Your "actually" says exactly what I said. :P Unless you meant to say you wouldn't be surprised if flat were slower.
I shouldn't have more than about 50-100 documents per folder max. But anyway, still a total of 600 and not quite instantaneous but close enough (a few 100 ms). |
02-04-2020, 04:02 PM | #23 |
BLAM!
Posts: 13,477
Karma: 26012492
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
|
@Frenzie: Oops, read that as "wouldn't" .
|
02-04-2020, 04:29 PM | #24 |
BLAM!
Posts: 13,477
Karma: 26012492
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
|
Yep, dot folders indeed: much faster on my end after https://github.com/koreader/koreader/pull/5816 .
|
02-05-2020, 01:16 AM | #25 |
Connoisseur
Posts: 57
Karma: 43710
Join Date: Jun 2008
Device: zaurus->palm->iPad->Sony PRS-T1,T2,T3->Kobo Forma&Likebook Ares
|
OK, so I am making some progress here (still working on the Likebook for the moment). I now rearranged the library on the device in the way suggested here and I now see the file manager screen coming up almost immediately, which is great!
Now I am experimenting with the calibre search, which I previously could not use, since it is only available on the file manager search menu, which took more than 10 minutes to appear. I did put part of my library on the SD-card, another part in the main memory. When I connect calibre, I can see all of them. On the first connect, it took about an hour to read the device, but now I am down to 2 minutes on every connect, so I am assuming Calibre uses a catalog on the device to speed things up. Calibre recognizes the main memory and the card separately and displays the list of books in there also separately. From the experiments I made so far is that KOReader's calibre search is using only the catalog on the main device, since I can not find books that are on the SD-card. Is that correct? Is there a way to instruct KOReader to use both catalogs? Or to switch between them? Or do I have to put everything on the main memory in order to be found? I just want to make sure I understand how things work before committing to one way or another. Thanks to all the help so far, Ctop |
02-05-2020, 05:36 AM | #26 |
Wizard
Posts: 1,613
Karma: 724945
Join Date: Oct 2014
Location: Antwerp
Device: Kobo Aura H2O
|
It looks like you can set two paths in your defaults.persistent.lua:
https://github.com/koreader/koreader....lua#L220-L226 Code:
-- Set a path to a folder that is filled by Calibre (must contain the file metadata.calibre) -- e.g. -- "/mnt/sd/.hidden" for Kobo with files in ".hidden" on the SD card -- "/mnt/onboard/MyPath" for Kobo with files in "MyPath" on the device itself -- "/mnt/us/documents/" for Kindle files in folder "documents" SEARCH_LIBRARY_PATH = "" SEARCH_LIBRARY_PATH2 = "" |
02-05-2020, 06:39 AM | #27 |
BLAM!
Posts: 13,477
Karma: 26012492
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
|
Yep, you'll have to set *both* paths manually, as @Frenzie suggested.
The autodetection only picks up the main memory otherwise, as you've found out. |
02-05-2020, 11:09 AM | #28 | |
Wizard
Posts: 1,613
Karma: 724945
Join Date: Oct 2014
Location: Antwerp
Device: Kobo Aura H2O
|
Quote:
The difference lies between just providing a rough count and actually "scanning" the attributes, whether the file's been opened, etc. of the files in the subfolder. Just giving a count is orders of magnitude faster. Actual display could also be made rather significantly faster, at least for sorting methods like filename and date modified, but that'd require a more involved rewrite to grab such info more dynamically per page instead of all at once. Imo there's a very good reason that no one ever noticed this performance issue though, because like I said over a 1000 files in one folder seems fairly annoying to manage even when it's fast. (Or at least that's the case for the 10k files that cause a noticeable half-second delay on my laptop.) |
|
02-08-2020, 06:44 AM | #29 | |
Connoisseur
Posts: 57
Karma: 43710
Join Date: Jun 2008
Device: zaurus->palm->iPad->Sony PRS-T1,T2,T3->Kobo Forma&Likebook Ares
|
Quote:
All the best and thanks to everyone! Ctop |
|
02-08-2020, 08:43 AM | #30 |
Wizard
Posts: 1,613
Karma: 724945
Join Date: Oct 2014
Location: Antwerp
Device: Kobo Aura H2O
|
As an aside, I've since tracked down a few choice performance optimizations:
Just giving a count should be very fast now: https://github.com/koreader/koreader/pull/5819 That is, subfolders won't significantly slow down the folder they're in anymore. The most taxing part of the code for displaying folders is ideally ~20% faster: https://github.com/koreader/koreader/pull/5827 The percentage is based on my test with 10k files. Also the performance improvement is less for the whole, that's just for that specific loop. But that loop is one of the slowest parts. |
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
How large is your Calibre library? | calibrejunkie | Calibre | 78 | 07-30-2021 07:25 PM |
KOReader doesn't open large, complex ebooks? | SbAgOoTs | KOReader | 33 | 01-13-2020 12:40 PM |
KoReader folder bug see on Nickel library | anki | Kobo Reader | 4 | 12-28-2019 07:24 PM |
Creating a dedicated library partition for koreader. | Markismus | Kobo Developer's Corner | 6 | 09-02-2013 09:02 AM |
Best way to get a large Calibre library into the PE library? | Filark | enTourage Archive | 0 | 04-20-2011 10:18 PM |