Libraries starting with Non-Latin Characters

HAL.Unke · 05-25-2020, 09:37 AM

Libraries/Folders are sorted alphabetically in the quick switch list.
One can move them to the end of the list by starting the name with z (e.g. z_boring_books).
I thought it would be nicer to use unused unicode letters (e.g. 􏿽test),
but if I edit author or title I get an error
calibre, version 4.14.0 (also with 4.17-64bit)
ERROR: Failed to set data: Could not set data, click Show Details to see why.

Traceback (most recent call last):
File "site-packages\calibre\gui2\library\models.py", line 1122, in setData
File "site-packages\calibre\gui2\library\models.py", line 1197, in _set_data
File "site-packages\calibre\db\legacy.py", line 442, in set
File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
File "site-packages\calibre\db\cache.py", line 1122, in set_field
File "site-packages\calibre\db\cache.py", line 1136, in update_path
File "site-packages\calibre\db\backend.py", line 1621, in update_path
File "site-packages\calibre\utils\filenames.py", line 471, in delete_originals
error: (123, 'DeleteFile', 'Die Syntax f\xfcr den Dateinamen, Verzeichnisnamen oder die Datentr\xe4gerbezeichnung ist falsch.')

The texteditor np++ has no problems with the folder 􏿽test.

Why has calibre (or python) problems with non-latin unicode paths?
I use Win10/german language

theducks · 05-25-2020, 09:46 AM

I suspect this is your (known) problem:
https://manual.calibre-ebook.com/faq...r-smart-quotes

Avoidance is the simplest answer.

kovidgoyal · 05-25-2020, 12:59 PM

I dont see how that is possible. non-ascii character are stripped from the filenames used for books in the calibre library automatically. How are you getting that to happen?

BetterRed · 05-25-2020, 06:33 PM

Quote:

Originally Posted by kovidgoyal

... how are you getting that to happen?

Press F2 on an Author - e.g. 'Better Red', paste a high order char from clipboard (e.g, an Omega 'Ω') at start of name and press enter.

Despite the message the Author name changes to 'ΩBetter Red', and under the covers I get two author folders 'Better Red' and 'OBetter Red' with hardlinks in the book folders from what I assume is a half done move.

BR

kovidgoyal · 05-25-2020, 10:19 PM

The author name will change, the filenames should be ascii-ized however. I cannot reproduce on my linux machine, but will try on windows when I have a moment.

kovidgoyal · 05-25-2020, 11:44 PM

I tried those steps on my windows system as well, no errors. Anything else unusual you can think of? Are you running the python 3 calibre betas? Does it happen when starting calibre with --ignore-plugins?

BetterRed · 05-26-2020, 12:53 AM

Not py3, but the error doesn't happen with --ignore plugins.

These are the PIs I run

CalibreSpy.zip
Count Pages.zip
Drop Search Results.zip
EpubSplit.zip
Extract ISBN.zip
Favourites Menu.zip
Find Duplicates.zip
GetFileName.zip
Job Spy.zip
Modify ePub.zip
Quality Check.zip
Reading List.zip
Resize Cover.zip
View Manager.zip

The only PI that looked a likely cause was GetFileName, so I it knocked out. And hey presto, no errors.

The only other File Type PIs I know of are: Alf's deDRM tool and DaltonST's RIS extraction tool but I don't have either, nor the inclination to install either of them.

BR

kovidgoyal · 05-26-2020, 02:22 AM

You should report it in that plugins thread then.

BetterRed · 05-26-2020, 04:48 AM

Quote:

Originally Posted by kovidgoyal

You should report it in that plugins thread then.

Done

@HAL.Unke - do you use the GetFileName plugin?

BR

Terisa de morgan · 05-26-2020, 12:37 PM

Quote:

Originally Posted by kovidgoyal

You should report it in that plugins thread then.

I'm able to reproduce the error... Without any FileType plugin (that includes GetFileName), so there is something more. What could I check? If I start calibre in debug mode, I get this trace:

Spoiler:

kovidgoyal · 05-26-2020, 12:39 PM

Post the exact set of steps you did to reproduce

Terisa de morgan · 05-26-2020, 01:55 PM

Copied from @BetterRed's post, updating it with my test data:

Quote:

I created an fresh library called 'ΩTest'

Using the Add Empty book option, I added a book with Author 'Terisa' and Title "Prueba 1', I checked the create EPUB option.

Then I F2'd the Author cell and pasted the 'Ω' at the front of the Author cell and pressed Enter, the error "Failed to set data: blah-blah" popped, but when I dismissed it I had 'ΩTerisa' in the Author cell. (The stack trace happens at this point).

I opened the Book folder and checked that they were hardlinks. Then I checked I had 'Terisa' and 'OTerisa" Author folders (the 'O' being the Latinised Ω), and there was a 'Prueba 1 (1)' folder in both Author folders and in the two book folders there was a hardlink to the epub file, viz:

Code:

ΩTest
    Terisa
        Prueba 1 (1)
            Prueba 1 - Terisa.epub (hardlink)
    OTerisa
        Prueba 1 (1)
            Prueba 1 - OTerisa.epub (hardlink)

This happens at the three situations (with plugins, starting calibre with --ignore plugins and removing files from the plugins folder).

DaltonST · 05-26-2020, 02:03 PM

Auto-add action from specified folder at startup?

The original file is deleted.

Terisa de morgan · 05-26-2020, 03:46 PM

Quote:

Originally Posted by DaltonST

Auto-add action from specified folder at startup?

The original file is deleted.

No auto-add, creating empty book (with epub) from the user interface. When the author name is changed, I guess a new folder is created and the original one should be removed, but it looks like there is a problem.

HAL.Unke · 05-26-2020, 04:54 PM

It is easy:

1. Step: Create a folder with Windows Explorer

Click image for larger version

Name: pic1.png
Views: 125
Size: 4.1 KB
ID: 179534

2. Step: Create library

Click image for larger version

Name: pic2.png
Views: 134
Size: 60.1 KB
ID: 179535

3. Step:Libraries are ready

Click image for larger version

Name: pic3.png
Views: 134
Size: 71.6 KB
ID: 179536

4. Step: Import book and rename author and/or title

Click image for larger version

Name: pic4.png
Views: 113
Size: 81.5 KB
ID: 179537

After clicking 3 or 4 at error messages, the change is accepted but the old versions are not deleted.

I use in this demo as first letter
⴮ ⴮reserved
፡ ፡ETHIOPIC WORDSPACE
�� 𒍮CUNEIFORM SIGN ZUM

@BetterRead: No, I don't use a plug-in 'GetFileName'. I deleted all user-installed plug-ins and a lot of the other. Did not help.

__
I hope the method of how I inserted the screenshots works. Otherwise, please tell me how to do it.

05-25-2020, 09:37 AM	#1
HAL.Unke Member Posts: 12 Karma: 10 Join Date: Jun 2017 Location: In the middle of nowhere in Germany Device: Android Tablet	Libraries starting with Non-Latin Characters Libraries/Folders are sorted alphabetically in the quick switch list. One can move them to the end of the list by starting the name with z (e.g. z_boring_books). I thought it would be nicer to use unused unicode letters (e.g. 􏿽test), but if I edit author or title I get an error calibre, version 4.14.0 (also with 4.17-64bit) ERROR: Failed to set data: Could not set data, click Show Details to see why. Traceback (most recent call last): File "site-packages\calibre\gui2\library\models.py", line 1122, in setData File "site-packages\calibre\gui2\library\models.py", line 1197, in _set_data File "site-packages\calibre\db\legacy.py", line 442, in set File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 1122, in set_field File "site-packages\calibre\db\cache.py", line 1136, in update_path File "site-packages\calibre\db\backend.py", line 1621, in update_path File "site-packages\calibre\utils\filenames.py", line 471, in delete_originals error: (123, 'DeleteFile', 'Die Syntax f\xfcr den Dateinamen, Verzeichnisnamen oder die Datentr\xe4gerbezeichnung ist falsch.') The texteditor np++ has no problems with the folder 􏿽test. Why has calibre (or python) problems with non-latin unicode paths? I use Win10/german language

05-26-2020, 12:53 AM	#7
BetterRed null operator (he/him) Posts: 21,667 Karma: 29711016 Join Date: Mar 2012 Location: Sydney Australia Device: none	Not py3, but the error doesn't happen with --ignore plugins. These are the PIs I run CalibreSpy.zip Count Pages.zip Drop Search Results.zip EpubSplit.zip Extract ISBN.zip Favourites Menu.zip Find Duplicates.zip GetFileName.zip Job Spy.zip Modify ePub.zip Quality Check.zip Reading List.zip Resize Cover.zip View Manager.zip The only PI that looked a likely cause was GetFileName, so I it knocked out. And hey presto, no errors. The only other File Type PIs I know of are: Alf's deDRM tool and DaltonST's RIS extraction tool but I don't have either, nor the inclination to install either of them. BR Last edited by BetterRed; 05-26-2020 at 12:55 AM.

05-26-2020, 04:54 PM	#15
HAL.Unke Member Posts: 12 Karma: 10 Join Date: Jun 2017 Location: In the middle of nowhere in Germany Device: Android Tablet	It is easy: 1. Step: Create a folder with Windows Explorer 2. Step: Create library 3. Step:Libraries are ready 4. Step: Import book and rename author and/or title After clicking 3 or 4 at error messages, the change is accepted but the old versions are not deleted. I use in this demo as first letter ⴮ ⴮reserved ፡ ፡ETHIOPIC WORDSPACE �� 𒍮CUNEIFORM SIGN ZUM @BetterRead: No, I don't use a plug-in 'GetFileName'. I deleted all user-installed plug-ins and a lot of the other. Did not help. __ I hope the method of how I inserted the screenshots works. Otherwise, please tell me how to do it. Last edited by BetterRed; 05-26-2020 at 06:34 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Latin non-English characters in titles	hfpop	Kobo Developer's Corner	11	05-29-2018 05:17 PM
In the file structure the name of the books are only saved with latin characters	jaimesuez	Library Management	3	06-30-2014 09:06 PM
Code view displays square box instead of non latin characters	mashru	Sigil	10	08-20-2013 11:09 AM
Problems inserting latin characters	Pandinus	Sigil	8	07-14-2013 02:42 AM
mobi conversion loses latin-extended-additional unicode characters	Gunaddho	Conversion	0	12-24-2011 02:52 PM

05-25-2020, 09:46 AM	#2
theducks Well trained by Cats Posts: 30,950 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I suspect this is your (known) problem: https://manual.calibre-ebook.com/faq...r-smart-quotes Avoidance is the simplest answer.

05-25-2020, 12:59 PM	#3
kovidgoyal creator of calibre Posts: 45,256 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I dont see how that is possible. non-ascii character are stripped from the filenames used for books in the calibre library automatically. How are you getting that to happen?

05-25-2020, 10:19 PM	#5
kovidgoyal creator of calibre Posts: 45,256 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The author name will change, the filenames should be ascii-ized however. I cannot reproduce on my linux machine, but will try on windows when I have a moment.

05-25-2020, 11:44 PM	#6
kovidgoyal creator of calibre Posts: 45,256 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I tried those steps on my windows system as well, no errors. Anything else unusual you can think of? Are you running the python 3 calibre betas? Does it happen when starting calibre with --ignore-plugins?

05-26-2020, 02:22 AM	#8
kovidgoyal creator of calibre Posts: 45,256 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You should report it in that plugins thread then.

05-26-2020, 12:39 PM	#11
kovidgoyal creator of calibre Posts: 45,256 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Post the exact set of steps you did to reproduce

05-26-2020, 02:03 PM	#13
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	Auto-add action from specified folder at startup? The original file is deleted.

Advert

Advert