Thread: Database Fork
View Single Post
Old 12-18-2013, 04:40 AM   #10
aleyx
Addict
aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.aleyx can self-interpret dreams as they happen.
 
Posts: 245
Karma: 20386
Join Date: Sep 2010
Location: France
Device: Cybook Diva
Quote:
Originally Posted by devils_add View Post
Guys, you are forgetting about the DMG file extension in Apple.
Dude, you're forgetting the concept of platform-agnosticism. I can install Calibre on pretty much anything. You're suggesting that only MacOS is worthy of Calibre?

Quote:
Originally Posted by devils_add View Post
Where everything the program needs to run is inside that file (which is an archive). So, what I am proposing is to have just the met file associated with record and record itself inside the archive which will be added into the main database.
Soooo... Everything in one file? That's even _worse_. To access your bit of database (Hah! bit! Get it? ^_^), you'll have to look into a big proprietary file then into a small .zip then into an XML? And that big proprietary file can be a small as a few dozen MB for a few books, up to several GB for consequent libraries? My own library is ~950MB and I only have about 1500 books. There's libraries out there with tens of thousands. I/O will be a nightmare.

Quote:
Originally Posted by devils_add View Post
The only time the archive is written to is when files are added or when metadata is changed,
Which is basically every time you use Calibre.

Quote:
Originally Posted by devils_add View Post
and all other times it is opened is to extract a needed file to read it or to send it to the device. Therefore, the main database file will be outside as it is right now. Also, you can have the main database link to virtual libraries databases for different, incompatible formats. In addition, this will allow for creating a single database for everything, with different iteration on front-end. So that you can have all your collections managed by just one database.
Sorry, I really can't picture it. Is it one database, or databases with "links to virtual library databases"? What are you calling "database" in this context? The way you talk about it, I think you mean "one big Apple-format archive in which there's one .zip per book, and in each .zip there's all the formats and one XML file describing the metadata for the book".

That's not a database, that's a .tar backup.

If that's not it, you really really need to do some kind of ASCII art of your file hierarchy, because right now I'm in the dark.

Quote:
Originally Posted by devils_add View Post
Sorry, for wordiness.
I'm really sorry, but the problem here is not the words, it's the concept.

Quote:
Originally Posted by devils_add View Post
Also, with the archive architecture, you can keep some pdf books broken by chapter, and combine them on the fly as requested (resources available), so you don't have to download the full book, but just the chapters you need.
First rule of arch rewrite: aim for feature equality, _then_ build upon it. Calibre's granularity doesn't go down to the chapter. That's why it can't do that. If it did, it would.

Also, the ressources you may (I say _may_) save with that system are utterly dwarfed by the ressources you'll use just to read your database.

There's no two ways of doing database-driven file management on consumer hardware. There's only one. There's one system, made of an RDBMS on one side and a filesystem on the other. Some of those files, if they're text files (as opposed to binary files) can be compressed, but that's as far as it can go.

I know that because I've already tried it all, ever since I've first discovered databases twenty years ago. Your system? I made one like that, more or less, when I was 16. At the time it was a VB4-based management system for the fanfiction I downloaded from R.A.A.C. (ah, those were the times...), and trying to make Eyrie Production's Undocumented Features into some sort of reading order with bookmarks, because even back then it was _huge_. It took me a few weeks before I scrapped the idea of text files-based DB and turned to Access (There was no SQLite in those far away times...). I had much better results.

So, learn from the mistakes of an old (well, 36-years old) pro and use an RDBMS. That's why they're made for. They're good at it.
aleyx is offline   Reply With Quote