Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 08-19-2011, 05:42 PM   #1
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none
Question Possible bugs with new "Languages" field.

Hi, I do really like this new feature.

Moreover, I did have in my Calibre database two custom fields for language: the first one for the current book language (#language) and the second one for the original language of the book (#original_language).

But, although I'm not sure I think there are some bugs with this feature.

1st one. Select "Greek, Ancient (to 1453)"
Voila you get an error. I think the problem is that it should be "Greek; Ancient (to 1453)" like "Spanish; Castilian".

2nd one. Comparison with the new Languages field doesn't work
Because of the new Languages field, I intend to erase my previous #language column, while keeping #original_language one.

In addition to #language and #original_language, I have got a calculated "Original Language?" field (with a "#original" lookup name).

The template for this field is
Code:
{#original:'strcmp(field('#original_language'), field('#language'), 'No', 'Yes', 'No')'}
With this code, it perfectly works (see Old-Right.jpg screenshot)

But if I update the #original field template, using Languages instead of #language, it stops working.

So this code:
Code:
{#original:'strcmp(field('#original_language'), field('Languages'), 'No', 'Yes', 'No')'}
doesn't work, (see New-Wrong.jpg).

Am I doing something wrong? or is it a bug?
Attached Thumbnails
Click image for larger version

Name:	Old-Right.jpg
Views:	271
Size:	620.1 KB
ID:	75579   Click image for larger version

Name:	New-Wrong.jpg
Views:	238
Size:	698.8 KB
ID:	75580  
arspr is offline   Reply With Quote
Old 08-19-2011, 07:43 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
1) There is indeed a problem with language names that have commas in them. Fix for that will be in next release.

2) The field name is languages with a lowerwcase l. Also, behind the scenes, the languages field does not actually store language names, rather it stores 3 letter ISO 639 language codes. So for strcmp to work your original language field would also need to store the ISO 639 codes.
kovidgoyal is offline   Reply With Quote
Advert
Old 08-20-2011, 02:54 AM   #3
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none
Wow, thanks for your reply.

About 2). Confirmed. I've just used eng instead of "English" and spa instead of "Spanish; Castilian" within #original_language and it perfectly works.

But with this situation, it would be really nice if you could add a new possible "Languages"-type column in the "Add your own columns" section. I mean from a plain user point of view, it's quite odd storing eng, spa, or whatever other needed code.
arspr is offline   Reply With Quote
Old 08-20-2011, 08:46 AM   #4
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by arspr View Post
Wow, thanks for your reply.

About 2). Confirmed. I've just used eng instead of "English" and spa instead of "Spanish; Castilian" within #original_language and it perfectly works.

But with this situation, it would be really nice if you could add a new possible "Languages"-type column in the "Add your own columns" section. I mean from a plain user point of view, it's quite odd storing eng, spa, or whatever other needed code.
Authors<->Books are Numbers assigned to Authors (names)
Series<->Books are Numbers assigned to Series (names)
Calibre has dozens of tables with key links

Relational DB's use keys for linkages. What you see on the screen is a very flattened view of your data and not a Flatfile like people make with Excel
theducks is offline   Reply With Quote
Old 08-20-2011, 11:47 AM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by arspr View Post
But with this situation, it would be really nice if you could add a new possible "Languages"-type column in the "Add your own columns" section. I mean from a plain user point of view, it's quite odd storing eng, spa, or whatever other needed code.
Why? As a user you will never see those codes, expect in a highly specific situation, like comparing to another column via a template. You are of course free to create whatever custom columns you like, but language in calibre has a well defined meaning, specified by ISO 639. The languages column will never accept arbitrary strings.
kovidgoyal is offline   Reply With Quote
Advert
Old 08-21-2011, 03:59 AM   #6
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none
Quote:
Originally Posted by kovidgoyal View Post
Why? As a user you will never see those codes, expect in a highly specific situation, like comparing to another column via a template. You are of course free to create whatever custom columns you like, but language in calibre has a well defined meaning, specified by ISO 639. The languages column will never accept arbitrary strings.
I need this kind of feature in order to keep my current data management. I not only store the book language but also the language it was written in, its original language. So I need two language columns. In addition I also perform a comparison between them which tells me if each book has been translated or not.

Till now, Calibre didn't have a specific language type of column, so I just used two custom Tag-like columns.

Now that Calibre offers a real and "official" language column, I do want to use this feature but:
  • I still need another extra language field.
  • If this second column is not available as a Language-type one, I'll have to use a general Tag-like column for it. But then, my current comparison between languages stops working, so:
    • Either I just don't use this new Language feature at all...
    • Or I have to store "spa", "eng", etc. in this custom second language column instead of their "true" names, which is completely awkward .
    • Or you provide some function within the template coding language in order to make a strcmp (My second language column (remember it's Tag-Like) vs. FunctionToConvertInternalCodesToDisplayedStrings(New Official Languages Column) )
    • Or there's another way to make this comparison that I don't know about. But this one was the solution that advanced users gave me (see this old thread about it)
arspr is offline   Reply With Quote
Old 08-21-2011, 11:50 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Feel free to open a bug report requesting a template function to translate from ISO codes to human readable strings.
kovidgoyal is offline   Reply With Quote
Old 08-23-2011, 11:29 AM   #8
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none
Thanks for your attention.

I've posted two feature requests, in case that some coder think interesting adding just one of them. (I DO really hope some coder think these possible enhancements are worthy ):

Keep working on this great software.
arspr is offline   Reply With Quote
Old 09-25-2011, 05:15 PM   #9
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none


Solved with 0.8.20

I've just tested the new language_strings(field(Languages), 0) function and it works like a charm.

I've just deleted my obsoltete #language column. Now I use the "official" Languages column, plus an extra #original_language.
arspr is offline   Reply With Quote
Old 09-26-2011, 06:43 AM   #10
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none
Sorry for the triple post...

Something is not working properly or probably I'm doing something wrong...

Look at the attached screenshot (at the selected book). As a test I've set languages as Latin (lat) and Spanish (esp) because I wanted to make a stress test (ISO alphabetical sorting is different from English names).

The trouble is that the "Original Language? (#original)" doesn't work fine. It should give Yes, but it returns No. I post its modified and updated code:
Code:
{#original:'strcmp(field('#original_language'),language_strings(field('languages'),0), 'No', 'Yes', 'No')'}
I firstly thought it was because of the different sorting (look at Languages and Original Languages columns). Because of it I made an auxiliary column "aux (#aux)" with just:
Code:
{#aux:'language_strings(field('languages'),0)'}
And apparently, the sorting is automatically fixed inside the language_strings function. Apparently #aux and #original_language are the same.

So, why does strcmp evaluate them as different? What am I doing wrong?
Attached Thumbnails
Click image for larger version

Name:	Clip.jpg
Views:	214
Size:	607.5 KB
ID:	76985  
arspr is offline   Reply With Quote
Old 09-26-2011, 07:08 AM   #11
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
The problem is that the order of the languages in the two lists are not the same. Both are in database order, which is roughly the order that they were added to the library. The language_strings function does not sort the list. You are seeing it sorted because the GUI sorts it when it is displayed.

Try
Code:
{:'strcmp(list_sort(field('#original_language'), 0,','),language_strings(list_sort(field('languages'), 0, ','),0), 'No', 'Yes', 'No')'}
chaley is offline   Reply With Quote
Old 09-26-2011, 08:01 AM   #12
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none


Perfect, sorting BOTH fields solves the issue.

(Before posting, I had tried sorting but only languages field. And I needed sorting #original_language too, because the database sorting thing you've just explained).

I've used a slight variant to your code:
Code:
{#original:'strcmp(list_sort(field('#original_language'),0,','),list_sort(language_strings(field('languages'),0),0,','), 'No', 'Yes', 'No')'}
I suppose that if I sorted before calling language_strings, I would get "Spanish, Latin" because of "esp, lat", which wouldn't match the alphabetically sorted "Latin, Spanish".

Last edited by arspr; 09-26-2011 at 08:10 AM.
arspr is offline   Reply With Quote
Old 09-26-2011, 10:55 AM   #13
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by arspr View Post
I've used a slight variant to your code:
Code:
{#original:'strcmp(list_sort(field('#original_language'),0,','),list_sort(language_strings(field('languages'),0),0,','), 'No', 'Yes', 'No')'}
I suppose that if I sorted before calling language_strings, I would get "Spanish, Latin" because of "esp, lat", which wouldn't match the alphabetically sorted "Latin, Spanish".
You are correct. The code I supplied would not work for that reason.

Why do you start with "{#original:"? That field is never used in the functions (the token $ does not appear). You could use "{:" and make it clear that there is no field being passed into the functions.

I am considering adding a "list_equal" function, where two lists are considered equal if they contain the same items in any order (case insensitive compares). That would get rid of the list_sorts, which are superfluous in this case. One question: I would assume that both lists have the same separator, making the call something like
Code:
list_equal(list1, list2, ',', yes_value, no_value).
What do you think?
chaley is offline   Reply With Quote
Old 09-26-2011, 11:36 AM   #14
arspr
Dead account. Bye
arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.arspr ought to be getting tired of karma fortunes by now.
 
Posts: 587
Karma: 668244
Join Date: Mar 2011
Device: none
Quote:
Originally Posted by chaley View Post
Why do you start with "{#original:"? That field is never used in the functions (the token $ does not appear). You could use "{:" and make it clear that there is no field being passed into the functions.
A very simple explanation, I didn't know I could use that syntax...

I'm quite noob using templates and in the Calibre Manual all the examples start with {whatever_field: ...}

Quote:
I am considering adding a "list_equal" function, where two lists are considered equal if they contain the same items in any order (case insensitive compares). That would get rid of the list_sorts, which are superfluous in this case. One question: I would assume that both lists have the same separator, making the call something like
Code:
list_equal(list1, list2, ',', yes_value, no_value).
What do you think?
I'm no expert, but this straight-forward comparison seems less hack-like than using strcmp.

Although, in order to prevent further work I would code (if easy):

Code:
list_equal(list1, list2, ','(for list1), ','(for list2), yes_value, no_value).
In this way you would have a fully robust comparison function fully independent from the separator.

Just one doubt. How will it behave with "Greek, Modern (1453-)"?
I feel that you have one serious inconsistency here. If languages_string gives the values separated with commas, commas shouldn't be used within the names.
arspr is offline   Reply With Quote
Old 09-26-2011, 12:23 PM   #15
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by arspr View Post
Although, in order to prevent further work I would code (if easy):

Code:
list_equal(list1, list2, ','(for list1), ','(for list2), yes_value, no_value).
In this way you would have a fully robust comparison function fully independent from the separator.
That is what I was thinking. I don't want to introduce parameters that have no purpose, but your argument below solves that.
Quote:

Just one doubt. How will it behave with "Greek, Modern (1453-)"?
I feel that you have one serious inconsistency here. If languages_string gives the values separated with commas, commas shouldn't be used within the names.
This is well-and-truly a problem. Not only will the proposed list_equal not work -- none of the current list functions work either.

My opinion is that the language names should not have commas in them. I would instead use semicolons. Unfortunately the language names come from elsewhere, and we have no control over them. For example, the German translation uses semicolons where all the other languages use commas!

I am not sure what to do. One possibility to add an argument (or change the current second argument) to language_strings, telling it to convert commas to semicolons. This will make the comma separator work correctly, but will have the side effect of making the results of language_string be different from that shown in the language column. Your 'system' would work only if you used semicolons instead of commas in your #original_language field. I would also need to make the language_codes function accept either style.

Another solution would be to preprocess the language code tables to convert all commas to semicolons. My guess is that Kovid would not be in favor of this approach, but I haven't asked him.

It might be that the right solution is the one that I don't want to do: a custom column that knows the contents are language codes. My problem is that creating a new column type has *many* ramifications throughout calibre, with a high probability that I will seriously break something.

More thought is required.
chaley is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Can't clear "custom" date field? joegearhart Calibre 5 12-20-2010 03:32 AM
Bug? "Insert metadata as page at start of book" doesnt encode Comments field properly rollercoaster Calibre 2 04-24-2010 10:40 PM
Rebuild "author sort" field enriquep Calibre 2 07-24-2009 11:21 AM
Observations of Bugs which do not seem to be mentioned in "Gen3 Troubleshooting" James Bryant Bookeen 24 04-16-2008 06:38 AM


All times are GMT -4. The time now is 12:58 AM.


MobileRead.com is a privately owned, operated and funded community.