07-02-2011, 07:13 AM | #61 | |||
Guru
Posts: 999
Karma: 19985
Join Date: Dec 2008
Location: Netherlands
Device: iRex DR1000S
|
Quote:
@Mr. X.: can you point me to some documents about the format of the .metadata and .cover files from Calibre? Quote:
W.r.t. the extra files, mdbindex can be changed to remove the extra files once they are processed. (although this might give sync-problems with Calibre). Quote:
|
|||
07-02-2011, 07:18 AM | #62 | |
Guru
Posts: 999
Karma: 19985
Join Date: Dec 2008
Location: Netherlands
Device: iRex DR1000S
|
Quote:
|
|
07-02-2011, 07:24 AM | #63 | ||
Guru
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
|
Quote:
Adding id to model required changing some of the db queries and I wasn't sure enough it wouldn't break other parts of the CTB code. Quote:
Code:
gint64 ermetadb_get_file_id(erMetadb thiz, const gchar* filename, const gchar* dirpath); Wait a couple of days, I won't be able to work on it this afternoon, but I'm close to finish the code to delete files from ctb. I think that's the last work to be done on the DR code. Anyway you can look at the last patch I published in this thread. It lacks delete files handling and some minor last-time fixes, but the other parts are finished. I expect to push the final version to BB tomorrow. Last edited by Iņigo; 07-02-2011 at 07:31 AM. |
||
07-02-2011, 07:41 AM | #64 | |
Guru
Posts: 999
Karma: 19985
Join Date: Dec 2008
Location: Netherlands
Device: iRex DR1000S
|
Quote:
|
|
07-02-2011, 08:32 AM | #65 | |
Guru
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
|
Quote:
What I'm doing is: - run "find /path" - parse the lines building a list of file - split in dirname, basename - get file_id - remove covers files As I've said, I'll finish it tomorrow as I have to go out in half an hour, and then push to BB. |
|
07-02-2011, 10:35 AM | #66 | ||
Connoisseur
Posts: 68
Karma: 64
Join Date: Mar 2010
Location: Philadelphia
Device: iRex DR800SG
|
Quote:
Quote:
If you put all thousand of your books in a single directory the 3x number of files would probably be trouble on FAT32, but calibre puts them in /Books/Author/ (and I organized my books similarly before calibre came along) which limits the number of files in any single directory. |
||
07-02-2011, 10:40 AM | #67 | |||
Connoisseur
Posts: 68
Karma: 64
Join Date: Mar 2010
Location: Philadelphia
Device: iRex DR800SG
|
Quote:
Quote:
Quote:
|
|||
07-02-2011, 05:25 PM | #68 |
Guru
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
|
Ok, code is now fully functional and finished.
You can find it in bitbucket repository, revision 55. Next steps are: - finish migration tool to work in both directions: fs2db and db2fs - adapt "icons" tool to handle in-filesystem thumbnails - write some documentation - adapt DR800+ installer to run migration tool, both for installation and also for uninstallation - release a new version of DR800+ Iņigo |
07-03-2011, 02:59 AM | #69 | |
Connoisseur
Posts: 71
Karma: 592
Join Date: Aug 2010
Device: irex dr800sg DR1000S
|
Quote:
One thing: I use an ereader a lot and it happens when a battery stop working in middle of page turn. Usually it happens in the middle of page turn. In such case a data base going to be corrupted and you have to delete it and recreate all thumbnails. A lot of time You know. My solution is - to keep a backup of database on the same place, so every time I add/delete files from SD-card I also copy a global.db to global_.db. And if global.db is corrupted I use a working version of global.db. The problem with your solution: it is hard to backup/restore a database if it not a one big file. I'd like an option to use/or not to use different thumbnails handling. Sincerely Yura |
|
07-03-2011, 03:24 AM | #70 |
Connoisseur
Posts: 71
Karma: 592
Join Date: Aug 2010
Device: irex dr800sg DR1000S
|
Additional problem, we almost forgot about, is indexing.
How it works: 1) At start of indexing it select from data base all files on the SD-card and create linked list of them(Huge memory load). 2) Then it creates linked list of all files on SD-card (Huge memory load) 3) Then for each file on disk it looks for (very slow) corresponding file on database list. 4) Compare it and mark if they different/missed/new 5) For every entry in data base list it apply changes if any. So to speed up indexing, one should 1) Limit count of items in memory. Limit memory load. 2) speed up search items in database list. IMHO it's useless to select all items from data base and then search it this list, database does this faster. 3) Limit count of file items in memory: limit memory load from file list. Sincerely Yura |
07-03-2011, 03:54 AM | #71 | ||
Guru
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
|
Quote:
Anyway, it's not a big issue either. As you make a backup of global.db, you can make a backup of System/.covers directory as well. Quote:
A runtime option would make code too complex even with current changes only. And providing two sets of binaries is very time consuming for me. Anyway I've included a compilation flag in the code (DR800_ENABLE_NODB_COVERS in ermetadb and ctb) to enable or disable it so anyone could build their own binaries if they want. Iņigo |
||
07-03-2011, 03:58 AM | #72 |
Guru
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
|
|
07-03-2011, 04:41 AM | #73 |
Addict
Posts: 326
Karma: 6908
Join Date: Sep 2010
Device: DRs, EE, ipad/iphone, Helix
|
|
07-04-2011, 12:26 PM | #74 |
Guru
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
|
In case someone missed it, I've just released v4 of DR800+ with the feature I've been working and commenting here: store thumbnails in file system instead of in global.db.
It's much faster with many books (> 500-1000) than the original firmware. |
07-04-2011, 05:30 PM | #75 | |
Connoisseur
Posts: 68
Karma: 64
Join Date: Mar 2010
Location: Philadelphia
Device: iRex DR800SG
|
Quote:
Code:
{ "rating": 0, "author_sort": "Lessig, Lawrence", "application_id": 238, "pubdate": "2006-02-15T05:00:00+00:00", "series": null, "author_sort_map": { "Lawrence Lessig": "Lessig, Lawrence" }, "publication_type": null, "size": 4334892, "author_link_map": { "Lawrence Lessig": "" }, "uuid": "1edb160e-2306-45a7-8fdb-e204b5213a33", "title": "Code 2.0", "user_categories": {}, "comments": "<p class=\"description\">SUMMARY:<br>There\u2019s a common belief that cyberspace cannot be regulated-that it is, in its very essence, immune from the government\u2019s (or anyone else\u2019s) control. Code, first published in 2000, argues that this belief is wrong. It is not in the nature of cyberspace to be unregulable; cyberspace has no \u201cnature.\u201d It only has code-the software and hardware that make cyberspace what it is. That code can create a place of freedom-as the original architecture of the Net did-or a place of oppressive control. Under the influence of commerce, cyberspace is becoming a highly regulable space, where behavior is much more tightly controlled than in real space. But that\u2019s not inevitable either. We can-we must-choose what kind of cyberspace we want and what freedoms we will guarantee. These choices are all about architecture: about what kind of code will govern cyberspace, and who will control it. In this realm, code is the most significant form of law, and it is up to lawyers, policymakers, and especially citizens to decide what values that code embodies. Since its original publication, this seminal book has earned the status of a minor classic. This second edition, or Version 2.0, has been prepared through the author\u2019s wiki, a web site that allows readers to edit the text, making this the first reader-edited revision of a popular book.</p>", "languages": [], "user_metadata": {}, "thumbnail": [ 45, 68, "/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAoHBwgHBgoICAgLCgoLDhgQDg0NDh0VFhEYIx8lJCIfIiEmKzcvJik0KSEiMEExNDk7Pj4+JS5ESUM8SDc9Pjv/2wBDAQoLCw4NDhwQEBw7KCIoOzs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozs7Ozv/wAARCABEAC0DASIAAhEBAxEB/8QAGwAAAgMBAQEAAAAAAAAAAAAAAwUABAYCBwH/xAA2EAACAQMCBAIIBAYDAAAAAAABAgMEBREAEgYTITEiQQcUMlFhcYGRFTNCoRYXUoKx0TSiwv/EABcBAQEBAQAAAAAAAAAAAAAAAAIBAAT/xAAgEQACAQQCAwEAAAAAAAAAAAAAAQIREiExQVEiMvBh/9oADAMBAAIRAxEAPwDOwpbdrNUVDbmVNoXcSrZ8ZP74799FjFkDbTJIwz0Z9wyMZ64+3011SWCGS2rcbndIrbBO7LT74mkaUjucL2UHpnRbNwZcb00jQT00VOjELPK5CyY816ZI6d+mu5tdnCovoqEWtkTYX3jHMyW2/pyV/wC3f/WuI1oTWPzJVWJZlwMt1TrnHn7tMazhe8WaupqJ1p5FrJVijqEYlCzdAGPcd89RoknB8tQ9THRXa31ddT7jJRwlw/h9rbuAyRqXLszi+hbGtuLLvkVeoz4nwMN1/Y/THxzr4BTAqYHVgY1yCB4T1/qP+Onz1ZoOHYaq0R3Oru9NQRSTNComR2JZQCfZHx0KSlp6CpaOmuUFcrKGMkGVAOT0OSpz/vSqg52WOLf+LYRH+ULWm3HbO5t3769FtVWv4FbvVWCxerIAIx8Ovkeuc682pb7S/hsdvutqW4R0zMaZxM0TR7jkqSO65640Cj4iuNvkk9TdIoXcsIAuUT5A9R99FR4Y7qZR6rX1kb09MlTsJasgEZPfdzFxjXnVDz/5mL6tnm/irYx7uYd30xnS9uIrhPdaSvqpOcaWZZUi9lcgg+Xy76Yz8YKslRPbLLSW+rqd3MqkdpJBu9rbnopOe41nHOEa6qyxtcHsS8Psa+Kqkpzearkequq47d8+WNZkmjarmNtSpSmwu1Z2LMOnXJAx31Zt/EVNS2ZLZW2WCvjjmaZGkmdCpYAH2floU9VSV9QXorXHQIqhWSNmk3HJ6ksCf8asU0GTTK0VsMtA9Y1SkSKSAro+WYfpBxgn/HnjV7hnhKt4mklaKSOmpYPzaiX2VPuHvOu6Lmm2xoq1LRFH3Kgmy3VvYKnZjp5+Y699UFvdeLH+CrNtozNzWVRgsegwT5jscavk9E8VsfXv0dXC125rjSVcFxpoxudoRhlHmcdcj5HWattuqbtcIaGjj5k8zYUeXxJ+A1vvRkZqKy3mvq2KWxY/1+yWAO7H0wPrpd6OYxTxX28ABXoqI8sn9BIJ/wDOPvo3NJ14Ham1Tkh9HtG0jUMHFFFLdV6eq4wC39Oc5z9Pp5azC0stFV1FLVx8qaFtjo+0FSM+/p9uny11FimkWqSQCaJuYJGYZ3A5yfF7+/3HnrWekGm5fEEVdFHKBXUscj8sBhuGR5/DHz1atOjzUOGqrBn6aGSG2x1fK7A7w9RtWNcsfCM5BJCr/affqcJcNS8SXhaRWKU8I3zye5QQMD4naca7oAFoEIkrJOV+fHA25cgs7lgHyAcqM48vjracH2et/lm5tmxa+5AkyO23C5x3+WcfPUlK1M0Y3NCLjfialaFOGrKBFbKTCSsnaQjb0B8wAc58zonAjMeEuKY1A5vq24Z8ztcEfcH76UXXgK82WlhlrDTBJpUp12S58TJtHl2yBp7ZaH+D+JKiyXmqhMF2ptjSxt0UvkDOe3iDD+4ajtspFiV19ZL5mLkVxRleaFRUBBO7oPLPl0PhPwxra+kRN5symlMrLRDOx8Y7f60CL0d31ritLVRQpRIwaSr5oKsufGNvfqBnt0Ol/F94ob1xNO8c7JTU8aQ08kZysijOSPqTqtqTVCJOMXXkSkSmg5PrEojMMjlVbAJZupOO/wBdGTnxSCOOsqURZSAqykAAJ7tTU01oEuQZ500cay1dTKoEDYeQsM5PXroclKswPNllcmOVMs2ThTkfbU1NZaM9lqasuDUkkLXOsMSpE4jMxKg+Yx7vhotst9Pz6qFl3JG4KBseHPU41NTSW/v0EvV/dH//2Q==" ], "db_id": null, "tags": [ "_type:Book" ], "timestamp": "2010-02-23T05:00:00+00:00", "mime": null, "authors": [ "Lawrence Lessig" ], "publisher": "New York : Basic Books, c2006.", "series_index": null, "lpath": "Books/Lessig, Lawrence/Code 2.0 - Lawrence Lessig.pdf", "language": "und", "rights": null, "title_sort": "Code 2.0", "cover": null, "book_producer": null, "identifiers": { "isbn": "9780465039142" } }, But the nook driver provides an example of how to put a higher resolution cover (or pair of covers) in the filesystem, and you could move those to .covers when indexing as you suggested if pulling the thumbnail from metadata.calibre turns out not to work or look good. |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Mobiperl Mobiperl - Perl tools for handling MobiPocket files | tompe | Kindle Formats | 890 | 01-17-2021 06:16 PM |
Large number of books on memory card | pwalker8 | Sony Reader | 8 | 03-24-2009 02:20 PM |
huge pdf files on iliad | yolle | iRex | 7 | 08-23-2008 10:59 AM |
Fictionwise Handling Fee Against Credit Card Terms of Service? | Gideon | News | 9 | 08-15-2008 02:13 PM |
Huge RTF files after conversion | TheMadBrewer | Sony Reader | 2 | 01-01-2007 02:55 AM |