Problems with very huge collection in Arabic

[hsawires]

Originally Posted by BetterRed

ack - btw which OS do you use ?

BR

Windows 7 64bit
Calibre Protable

[DoctorOhh]

Originally Posted by hsawires

I post you a reply, but it have to be approved by the moderator, probably because I attached an Image to my post.

Temporarily approved the post, but the image is too large. Please edit the post with a smaller version of the image.

[hsawires]

Originally Posted by DoctorOhh

Temporarily approved the post, but the image is too large. Please edit the post with a smaller version of the image.

Thank you for editing my post.

[BetterRed]

Originally Posted by hsawires

not at all please ask whatever you like. I am appreciating your help so much.



yes without any problems. you can see a screeshot of one of my books.


redundancy my friend, HD spaces, duplication of the same file. I think it is an important issue, specially when i decided to use a data management software.



I am afraid that I couldn't understand your point.

If you mean that storing the books in the Library in whatever manner will not affect using Calibre. yes this is true unless it make another copy of the file. and that lead me to the "duplication issue" again, I echo Sabardeyn in his last post. that I have to deal with two places storing the books, one for Calibre and the other is the safer one.


yes of course and no problem with that, except exporting a document to epub in Arabic have some problem some in encoding the characters, vowels and special character, and some issue with the direction RTL. but this is not my case because most of my e-books are (text on image) or image only PDF's. but if you like, it will be a pleasure for me starting a Thread about all the Arabic issues in Calibre.

The intent of my questions was to get some idea of what you want from Calibre - as I see it, there are at least three ways that Calibre can be used

Simply for doing conversions, downloading metadata and covers, embedding the metadata, attaching covers etc - but nothing is retained 'permanently' in a Calibre library. I think of this as a Calibre workbench.

Others do that and keep a record of their books in Calibre but they don't keep the actual book (format) files in Calibre, in this case they are using the Calibre database as a customisable catalogue. They would have the folders, but the only data would be and metadata.opf files and probably cover images.

And thirdly there are those who do all of the above and keep their book (format) files in one or more Calibre Libraries and very often have the library tightly coupled to their devices.

I don't think I could cope with having meaningless folder and file names if they contained my books. Whilst I don't write anything to or remove anything from my Calibre Library folders, I do expect them to have recognisable folder and file names. I sometimes reference them them in Calibre via the Insert Link feature in long text (Comments) custom columns. Another thing that would bother me is - how can I recover a book from my backup if I cant make sense of the folder and file names.

Amongst the 3.5 million active Calibre users there must be some who use Arabic or something similar - it would be interesting to know how they get along with the 'translated' folder & file names.

BR

[hsawires]

Originally Posted by BetterRed

The intent of my questions was to get some idea of what you want from Calibre - as I see it, there are at least three ways that Calibre can be used

Simply for doing conversions, downloading metadata and covers, embedding the metadata, attaching covers etc - but nothing is retained 'permanently' in a Calibre library. I think of this as a Calibre workbench.

Others do that and keep a record of their books in Calibre but they don't keep the actual book (format) files in Calibre, in this case they are using the Calibre database as a customisable catalogue. They would have the folders, but the only data would be and metadata.opf files and probably cover images.

And thirdly there are those who do all of the above and keep their book (format) files in one or more Calibre Libraries and very often have the library tightly coupled to their devices.

I don't think I could cope with having meaningless folder and file names if they contained my books. Whilst I don't write anything to or remove anything from my Calibre Library folders, I do expect them to have recognisable folder and file names. I sometimes reference them them in Calibre via the Insert Link feature in long text (Comments) custom columns. Another thing that would bother me is - how can I recover a book from my backup if I cant make sense of the folder and file names.

Amongst the 3.5 million active Calibre users there must be some who use Arabic or something similar - it would be interesting to know how they get along with the 'translated' folder & file names.

BR

I know Calibre is a workbench for conversion and most of the people using it as an e-book conversion tool, maybe I am expecting more from it.

as you may know there is nothing -- and I mean nothing -- about PDF's organizing and cataloging over the Internet, all available online are so naive .. Calibre is a great cataloging tool specially when it comes with conversioning and connecting with my e-book device. without mentioning dealing with RTL and extracting the cover. Personalty I take advantage of Calibre for cataloging database of my huge archive, I was thinking of hiring a coder doing the job as an offline web application.

the question that I dont get a logical answer on it, .. if calibre is able to store the files and folders with their real names, with the same exact metadata, when you export using save to disk, why it is so hard to do the same in its library folders?

as you said it is nonsense keeping files and folders on my hard disk with meaningless names !! maybe I will accept dealing with a huge one solid database that is encrypted and not accessible. and when it come to backup your library you have to keep your old folders and file location as well as Calibre library .. and make a backup of your huge calibre database. we are taking about tripling the collection. in my case I may need a 500 GB free doing the catalog.

Calibre is a great tool, It may need some tweaks and it can be universal having no competitor, and tell you the truth it deserve to be produced as commercial SW.

With pleasure, I will report all the Arabic issues in Calibre if it will enhance the final product.

[theducks]

Flip the thinking around.
Which way is easier to manage?

If you put a book on a shelf.
Is it easier to find:

if the shelf is in your house?
or
In a open access (more public) area?

In the user area of your disk(s), there many things (usually the user ) that can modify the file(name) and location.

Calibre would have to search the entire filesystem to rediscover books, THEN, try and match them to existing entries.

[hsawires]

Originally Posted by theducks

In the user area of your disk(s), there many things (usually the user ) that can modify the file(name) and location.

Calibre would have to search the entire filesystem to rediscover books, THEN, try and match them to existing entries.

Sorry. I dont get your point.

[theducks]

Originally Posted by hsawires

Sorry. I dont get your point.

My point is if you allow the files to be anywhere, Library (title) associations are easily broken. People have place their Library (folder) on the desktop . 1 Click later: 'Where did my books go?"

Why use a Library Manager at all?

What you seem to want is the conversion function.
For that, ignore the GUI and use the command line (CLI). there you get to specify where you want the output and where the input is.

[PeterT]

@TheDucks: I can see the other concern with calibre just indexing books and not moving them to it's own store, is how to ensure that you don't delete material at the O/S level that calibre still has catalogued.

[theducks]

Originally Posted by PeterT

@TheDucks: I can see the other concern with calibre just indexing books and not moving them to it's own store, is how to ensure that you don't delete material at the O/S level that calibre still has catalogued.

THAT was what I was trying to get across.
Calibre KNOWS where it put books (and the common advice is stay out, at least until you know what CAN be done with no harm). Anything else is a crap shoot.

[BetterRed]

@PeterT & @theducks - would you be happy if the names used for author folders, book folders, and format files in your libraries were gobbledegook - apart from the book-id in the book folder names.

What would your reaction be when "you know what CAN be done with no harm" and you summon the courage to press 'o' on your "Charles Dickens, Bleak House", and you're presented with what you see in the first attachment.

I know what I'd do, I'd ask - What the <expletive deleted> is this crock of ****.

Or maybe you don't even do that, maybe you click on a PDF in Book Details, then the plumber arrives - 30 minutes later you return to your computer to be faced with what you see in attachment 2.

BTW those incomprehensible file names are going into the MRU lists for Word, PDF Xchange Viewer and other programs such as Sigil or OOo Writer... So to pretend that the folder and file names are only relevant to Calibre is to ignore reality.

BR

[theducks]

BR this is not a discussion on obscuring file names (that only slows less knowledgeable folk ), this is about placing files into a controlled filing structure.

Personally, I like what BOINC did.

The files are owned by BOINC (the system user) service
The Logged user runs a GUI client that talks to the BOINC service .
If I take my boxes down to the local 'Public Storage', I don't get to put them in just any unit. I must use the assigned unit or con a buddy into using theirs .

[BetterRed]

Originally Posted by hsawires

the question that I dont get a logical answer on it, .. if calibre is able to store the files and folders with their real names, with the same exact metadata, when you export using save to disk, why it is so hard to do the same in its library folders?

Probably the only person who can give you a definitive answer is Kovid Goyal (see post # 32). I have no doubt that he would have given considerable thought to the decision to transliterate RTL Author Names and Book Titles into LTR English when creating the library folder and file names. Kovid is not someone who takes the easy way out.

My only thought is that some third party applications may not be able to handle RTL folder and file names - you'd know about that, better than me. When a book is saved to disk then Calibre loses sight of it - hence, from Kovid's perspective, its 'safe' to use the RTL character set.

I have another reason why I couldn't abide incomprehensible folder and file names - I use Windows Search and Recoll on my Calibre Library folders.

=================
So what would I do?

Firstly I should say that my main use of Calibre is as a 'reference' library in which I keep 'documents' such as government reports, consultancy and academic papers, journal articles, so its from that context I make the following suggestion.

I would explore the use of Calibre as a conversion and metadata reference (cataloguing) tool. I would keep the 'permanent' format files in a parallel folder hierarchy - ie one based on Author Names and Titles using RTL Arabic. This folder hierarchy ('content library') would be 'maintained' via Calibres Save to Disk feature. So books would be imported into Calibre for the purpose of 'deduping', getting the author and title properties consistent, extracting covers, updating metadata and conversion. I would then export them to the 'content library'

So how would I get from an item in the Calibre book list to the book format files. I would insert links into a custom comments column to the equivalent 'content library' folder, and maybe the format files too. The custom column and its links would be show in Book Details and the links to the content folder and format files could be clicked on to open them. Once I'd done that I would remove the format files from the Calibre library.

What would I lose. Firstly tight integration with devices, but that wouldn't bother me personally as I don't use it. Calibre would not know in what formats I had the books, but if that was really important I could record it manually in a suitable custom column. I'd lose the convenience of pressing 'v' to read a book, 'o' to open the content folder, I use keyboard shortcuts whenever possible, so for me these losses would be annoying.

I'm sure there are other things that I would lose but they may also be irrelevant to me. That's why I said I would explore the idea.

I'd have to be aware of the consequences of changing Author names and Book titles - especially Author names, but its not something I would anticipate doing carte blanche once I had exported the book.

I would maintain backups of the Calibre Library folders and the content folders via the same process - for me that would happen anyway, I use the same software and process to backup all my data.

I think I am right in saying that Arab writers often use pseudonyms (kunya names ?), is this an issue for you and are there any conventions on how to 'handle' them - i.e. what do Arabic libraries do?

BR

[BetterRed]

Originally Posted by theducks

BR this is not a discussion on obscuring file names (that only slows less knowledgeable folk ), this is about placing files into a controlled filing structure.

Personally, I like what BOINC did.

The files are owned by BOINC (the system user) service
The Logged user runs a GUI client that talks to the BOINC service .
If I take my boxes down to the local 'Public Storage', I don't get to put them in just any unit. I must use the assigned unit or con a buddy into using theirs .

That's not how I interpret the OP's issue. He seems to NOT be concerned about the filing structure.

IMO he is concerned about the labels used in the filing structure. If you read Kovids post you will see that he refers to the transliteration of RTL characters, not the filing structure.

If you use an RTL character set in Calibre then the folder and file names have no immediately obvious relationship to the author and title strings stored in the Calibre database.

I have no problems with accepting that "Les Misérables" in the book list will have a book folder name of "Les Miserables (51)", but if if was "51fd;owpy[devqyt (51)" then...

I have no idea what BOINC is/was/will be - maybe you mistyped BJORK

BR

[theducks]

Originally Posted by BetterRed

I have no idea what BOINC is/was/will be -
BR

Hint:
It is embraced at many World wide Universities doing research