01-17-2018, 09:21 PM | #1 |
Member
Posts: 23
Karma: 10
Join Date: Jan 2013
Device: iPad Pro 10.5
|
iOS 11 Smart Punctuation Fouls Content Server Searches
Huge fan of Calibre (3.15 Win x64), but I'm getting regular complaints about this...
iOS 11 (e.g., iPad/iPhone) introduced "Smart Punctuation" that auto-replaces straight quotes with "Smart Quotes" and made this the default setting. This is actually useful in many cases/applications, but the Calibre Content Server search function requires straight quotes as delimiters (e.g., title:"this and that"). iOS doesn't have the ability to insert by character code or a Character Map applet, so these aren't viable work-arounds. Further, since the Content Server interface is designed to be accessed by regular users (not necessarily technically sophisticated), suggesting regular expressions or other non-intuitive work-arounds that the user must remember isn't viable. I'd also prefer not to instruct all users to change a default setting on their iDevice just to access the Calibre content server. It'd be VERY useful if there was a tweak or Search preference that allowed the Search capability to treat smart quote delimiters as straight quote delimiters for users that desired. If it exists, I haven't found it yet. Any suggestions? Last edited by skipster10; 01-17-2018 at 09:25 PM. |
01-17-2018, 11:16 PM | #2 |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Sigh. Trust Apple to implement something like this with no way to turn it off on a per input field basis in Safari.
Two possible solutions: 1) Just blindly replace smart dquotes in the search expression before sending it to the server. Simple to implement, but it breaks searching for terms that contain smart dquotes. 2) Actually modify the search expression parser to handle smart dquotes as quotation delimiters. Since, IIRC, the parser was written by chaley, in its current form, he should comment on how feasible this is. |
Advert | |
|
01-18-2018, 02:03 AM | #3 |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
@chaley: I just pushed an implementation of supporting smart quotes as quote delimiters. I'd appreciate it if you could review it.
|
01-18-2018, 05:53 AM | #4 |
Grand Sorcerer
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
@kovid: I am looking at it now and I am confident that it does what you intend, but the approach makes me nervous for several reasons:
I think it would be better to deal with the problem directly in the content server, replacing "smart" quotes if the user agent is an iOS (or perhaps any Apple) device. That way you can:
|
01-18-2018, 06:41 AM | #5 |
Grand Sorcerer
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
FWIW: The ICU tools include a transliterator called "Any-Publishing" that converts publishing punctuation to typewriter punctuation. For example, it converts ‘“foo”’ to '"foo"'. It might make sense to use it in a variant of "primary_contains" so searching ignores the accent types. I don't understand enough about how you did the ICU python bindings to know a) if this idea is feasible or b) how to do it if it is feasible.
It also might make sense to use something like the transliterator in the content server to process the search strings. |
Advert | |
|
01-18-2018, 06:44 AM | #6 | |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Quote:
Last edited by kovidgoyal; 01-18-2018 at 06:46 AM. |
|
01-18-2018, 06:52 AM | #7 |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
I tried this on an iPad and you can type normal quotes by long tapping the quote button. So I dont think we need to do anything.
|
01-18-2018, 09:25 AM | #8 |
Member
Posts: 23
Karma: 10
Join Date: Jan 2013
Device: iPad Pro 10.5
|
Kovid:
Good find on the long-press. Since this work-around requires the end-user to be aware of smart-quote problem and how to workaround it, Content Server end-user education is required. I’m not a novice, but with my eyesight and the size of an iDevice screen, I couldn’t see the smart quotes and it took me a while to figure out why my searches weren’t finding known entries in the database when accessing from iPad. If the user doesn’t have prior knowledge of the database contents, they may not be aware that their search results aren’t reflecting as intended. My preference would be to solve the issue for the end user so no end-user knowledge is required, but if this workaround is the final answer, can the content server at least throw a warning and instructions to the user if the user agent is iOS and a search is entered with smart quotes? Thanks much for a fantastic tool that we use daily! Last edited by skipster10; 01-18-2018 at 09:31 AM. |
01-18-2018, 11:50 AM | #9 |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Sure, adding a warning is worthwhile. THough really, the fix is for apple to allow a website to specify that it does not want smart anything in an input field.
|
01-19-2018, 11:47 PM | #10 |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
|
10-08-2019, 09:17 AM | #11 |
Member
Posts: 23
Karma: 10
Join Date: Jan 2013
Device: iPad Pro 10.5
|
It seems that version 4.0 lost the ability to handle iOS smart quotes in searches. The prior solution of asking to replace smart quotes worked well.
Last edited by skipster10; 10-08-2019 at 09:20 AM. |
10-08-2019, 09:38 AM | #12 |
creator of calibre
Posts: 43,744
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
There weren't any changes to that code in 4, what exactly do you mean by lost the ability?
|
10-08-2019, 09:42 AM | #13 |
Member
Posts: 23
Karma: 10
Join Date: Jan 2013
Device: iPad Pro 10.5
|
Prior to 4, if a search on the content server contained smart quotes (i.e., iOS default), the content server detected the smart quotes and queried the user if smart quotes should be treated as regular quotes. With 4.0, no query is thrown and the search result just finds nothing.
|
10-08-2019, 10:05 AM | #14 | |
Resident Curmudgeon
Posts: 73,510
Karma: 126422064
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3
|
Quote:
https://www.howtogeek.com/344310/how...hone-and-ipad/ |
|
10-08-2019, 10:17 AM | #15 |
Member
Posts: 23
Karma: 10
Join Date: Jan 2013
Device: iPad Pro 10.5
|
Understood that it can be turned off. It is, however, the default setting for iOS. Thus, most users will have smart punctuation turned on and will not know how to turn it off. Furthermore, a search response with nothing returned is misleading when, in-fact, the issue is the smart punctuation. The former solution of prompting the user was an excellent solution that cautioned the end-user who didn't know any better. (Note: When upgrading to Version 4, even my first response was "Where'd all my books go?" when my searches started returning the null set. I'd gotten so used to the prompt...)
Last edited by skipster10; 10-08-2019 at 10:20 AM. |
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
3.0.0 Content Server and iOS version | TomCombs | Devices | 15 | 04-18-2018 11:16 AM |
Content Server View Books In Safari Issue With IOS 11 | theengine | Devices | 1 | 09-28-2017 05:50 AM |
Calibre Content Server Searches Page | r_sauve | Recipes | 7 | 08-08-2016 01:27 PM |
Accessing Calibre content server from iOS safari with username & password in address | Richs | Calibre | 0 | 01-18-2014 02:02 PM |
Trying to get to Content Server on OS X 10.7.2 using iPad 2 iOS 5.0.1 | cdg | Apple Devices | 0 | 11-15-2011 03:25 PM |