Universal Naming

[Philosopher]

Several years ago I was puzzled. Books in my library (the file - not the entry) would "disappear". It took a while to figure out - and it was the result of the naming of the files - and my accessing the database from different operating systems. The problem came when I loaded a book on a Mac or Unix system - but tried to access it on a PC.

I hope someone can help me SET a universal naming system on any of my Calibre installs that will be compatible with the Windows OS for future entries. And to take existing libraries created on a Mac or Unix system and bulk rename them to that system now.

It seems like this should be achievable - but I am just not expert enough in this area to figure it out. (Although I said this back when I first recognized this - I would love to see a SETTING in Calibre that would be a simple click to tell any install to use a Windows compatible naming protocol for users who don't understand this.

Anyhow - any help would be greatly appreciated.

[JSWolf]

The solution is do not access a Calibre database from more then OS. Pick which OS you will use and stick with that. Do not mix Windows, OS X, and/or Linux. Pick one and go with it.

[eschwartz]

The issue seems to be primarily one of case sensitivity. linux filesystems, and *some* OSX ones, are case sensitive, Windows is merely case-preserving.

My advice is, whenever switching to another OS, run Library Maintenance ==> Restore Database. Yes it will take a while, But it will make sure calibre can still see all your books.

You keep on switching between OSes and it's getting tiresome? Sorry, I've got no answer for you, other than... don't.

Choose one OS you will use, run the content server to access your books on another computer, and have done with it. IN a pinch, you can always transfer it over, as above.

One of these days, Kovid will eventually upgrade the content server to be able to fully manage calibre libraries (as in, write access). That should help things, but there is no ETA.

[Philosopher]

The problem with the "advice" of not switxching OS's - is simply not sufficient for my needs. I have several computers - and work on three platforms (Ubuntu, OSX and Windows). Sometimes I want to access the library from one of those - not necessarily the original used to create the database.

I am definitely not sufficiently knowledgeable about the file writing capacities and programmability of calibre to figure this out myself. I assume that there may indeed be a working solution to this - with manually setting up the parameters int he proper preferences. But I may be wrong (and if so - this is something I thinki someon including Kovid should be seeing as a very high priority - because the cross-platform abilities becoming "seemless" would be the most desirable "standard" (which could then be tweaked by more knowledgeable users (programmers, etc) but would ALWAYS default for the non-programming user. (Again - I am sure that this is Kovid's goal).

My understanding is that the problem that I encounter with cross-platform use is not the issue of case-sensitivity - but the issue of the LENGTH of filenames. Because Windows cannot "see" the file - if its name is longer (as Unix and thus also OSX have no problems with) - I think the default standard should be a windows data structure (since it will not cause problems in other systems - although a more advanced user in one of those platforms may prefer to take advantage of the lengthier file name opportunities.

Thus - when the unix (and thus OSX) filename assigned/written is LONGER than windows can handle - windows in Calibre cannot "see" the book (it looks as if it was missing - which when I first encountered this years ago - I mistakenly thouyght was the case - but the file is there - just without accessibility in windows. ANd worse - windows being what it is - it is EXTREMELY difficult to change that filename (or even delete the file) - altough I have found ways to do this (just not anything efficient).

I will check to see if a TEMPORARY fix option is provided in the idea of the database MAINTENANCE processes. This may profide a quicker fix for when this becomes very problematic - but its not an ideal long-term fix.

So - let me ask one one more time in this area for possible ideas (or the reason its not possible) that might work - in terms of is there a way to manually set up each of my Calibre installations with the SAME (but using the windows naming scheme for all) FILE LENGTH. Cutting off names autamatically that will give windows problems. It seems if this is not currently possible in Calibre - it MAY be something that could fairly easily be coded to make it possible (again I am not expert enough in the details of Calibre's internal commands on writing to file to be certain).

But also - I would really encourage serious discussion by those with expertise in this area to see if a fairly simple fix - or even perhaps a plugin method to do this - could be devised ... and eventually become incorporated into the standard release.

[eschwartz]

Like I said above, I believe case sensitivity issues pose a problem as well.

Anyway, I doubt it is a priority for Kovid, simply because it isn't all that common to switch between OSes with such frequency.

[Philosopher]

Yes - I recognize the case sensitive issue - although as far as I have experienced it - this does not seem to create the problem that I encounter with file length. It seems my windows can "read" the file - regardless of case (but I understand if CHANGING the case were being done - this would likely cause problems). But it is only when the file name length is too long - and you get that very annoying issue in windows where you can "see" the file (Calibre cannot even see it - i.e. the file size and format data are blank when viewing a library created in unix/osx in a windows install.). But windows cannot (and so Calbre - even if it saw the file - like the Windows file manager does - cannot open it either (or copy it, or alter it, etc.) So you can't in any way open the file (with Calibre or in windows generally - i.e. a PDF with a too long file name) - but worse - you cannot even get windows, directly, to even EDIT the filename down to a proper size. (That gets very frustrating).

That was why I was hoping that a few preference tweaks might be able to DELIMIT the default (I would do it manually in each install - but would suggest it be considered a starting default (alterable by user) for the base install of all platform versions/ BUt in effect forcing my calibre installs on unix/osx machines to use windows "friendly" file names.

It seems like it would be a simple task - if one knew the places it would need to be tweaked in calibre and what the formatting language needed would be. BUt again - there may be something writting into each of the calibre platforms that, at the moment, is "stuck" or fixed so it can't be done. I am HOPING that there may be a work around - but perhaps not.

If not - I do indeed think it is an important topic for this with more detailed knowledge and expertise of the filewriting code for Calibre may wish to discuss. Again - the ideal for Calibre - so far as I understand it (as it is an ideal for any program I believe) would be that its "standard" setup would be the most "friendly" and "usable" to those with NO programming or platform knowledge - while being configurable for those advanced users, knowing what they are changing, to fully utilize the file naming conventions in unix based systems that are not compatible with windows.

Thanks again for you comments and feedback. Let me know if I am clear in what I am suggesting and asking - as I think this is a useful topic for investigation/discussion.

[theducks]

I suggest using remote desktop tools and just using one System as the 'Main Library' when Management is needed.

Use the Content server for Content needs.

[eschwartz]

Well, the case sensitivity works the other way. Windows names a file x, and the calibre database asked it to create X -- works fine when windows tells calibre the file is there, but linux doesn't and can't find the requested file.

And the file length is already properly worked around on Windows, but longer filenames are allowed on linux. I don't know why, but it seems to be deliberate? Possibly because there is no need to restrict linux users from having more meaningful filenames when they aren't expected to be switching to Windows all the time.


Which isn't to say there cannot be a solution -- but I don't know how worth it it is for Kovid to spend time on something that can be almost guaranteed to be used by maybe like one person.


I would suggest that at least for the moment, Remote Desktopping would seem to be the easiest solution.

[Philosopher]

Couldn't I set a maximum string length - with strcat - in the save file preference?

[eschwartz]

That will only get applied to books saved with save-to-disk.

[Philosopher]

Ahh - thanks again (I am now understanding the issue a bit more as to why this is not as "simple" as I had assumed).

I will keep trying to think about work-arounds - but it appears I am at a dead end.

I do think it is important for the future development of Calibre to address this (but I see that perhaps this is so "deep" in the coding - it may be a "big task".)

I think in terms of developing the software - if the goal is to have it as "universally" accessible to the user (who I presume is primarily a mere reader (of books) and not a programmer) - this issue becomes of real importance.

I agree that the program should not LIMIT a unix-based user (and thus mac) from taking full advantage of the greater flexibility in naming that OS provides. I simply think that - in terms of useability outside of an expert community - that the DEFAULT saving and thus file naming practice should be the MOST UNIVERSAL (but alterable to the advanced user via preferences).

Thus I think it should ideally be designed such that any (default) file name should be IDENTICAL across platforms (and thus reduceable to that which is the most "limited" format). This way ANY user (of any degree of programming knowledge (including zero) on any platform could never encounter this problem (by default) and their libraries would be equally accessible no matter what plafform they are using. (A good scenario would be a Mac user who builds their library in that system - but then converts to a PC user. That user should NOT face the central issue I am addressing here (the default file saving and naming - is DIFFERENT by platform but most significantly - the windows user (I would assume the assumption being this being the majority users) is faced with the INCOMPATIBILITY issue (which does not occur in the other direction) - when the filename assigned in the unix system turns out to be too long for windows to "recognize" the file).

So my thinking is to seek to implement a standard whereby the assumptions can be:

1) A non-programming literate user (a simple "reader" of books);
2) A database created that (by default) can be read in ANY version of Calibre (Cross Platform) - and thus with the default set by the LEAST permissive file system - ie Windows.

It may be - however - that if this was not premised at the start of the project - at this point in time - to CHANGE this would be so GREAT a task that, at least for the moment, it is not something that would be viable or desirable to look into. Alternatively - if this is not as "deeply entrenched" - but could be subject to a more "simple" fix - I would highly recommend its being considered in a future update.

Thanks again for this discussion and your help.

[Philosopher]

Just to update - I did TRY the last suggestion about the stringcat function on my MAC installs - and so far I have NOT YET encountered a file length problem on my WINDOWS installs opening the same library.

I have not PUSHED THE LIMIT of this yet - to know if I am just getting "lucky" - testing all known past examples of when I got that problem. But thus far - and when I look at the filename that is stored it appears to be concatenating sufficiently (I can see that on the book title in particular - where this really becomes the issue primarily). But one thing I have not checked - is if I have a multi-layered (subdirectory intensive) file structure - where I also know this has a LIMIT in windows - that can really add up and use up the allowed length.

But my fingers are crossed - that this may indeed have worked. I'll update as I keep my eye on this.