how to handle unicode chars in filenames in python?

[At_Libitum]

Hi,

well, maybe not exactly unicode, more like accented characters, so utf-8 at a minimum

I'm hitting the a wall, I need to find a function that converts this

Götterdämmerung

into this

G\xf6tterd\xe4mmerung

I've been all over the source but cannot find how it's done in python....

[kovidgoyal]

Trying to put non ascii characters into filenames robustly is impossible, there is a reason calibre itself convert all filenames to ascii. Different kernels and different filesystems encode filenames in different ways. And on some filesystems, these ways can depend system settings.

That said if you are doing it in a limited context and know what encoding to use then just do

filename.encode('utf-8')

assuming filename is a unicode object and not already a bytestring. If it is alreadya byte string then youneed to know what encoding it is in and decode it first like this

filename.decode(encoding).encode('utf-8')

[At_Libitum]

Sadly I cannot avoid using them because the reader uses the author-title combo as actual filenames.

.encode('utf-8') gave me the same as went in, so left the ö and such as-is

and turns out that format(json.dumps(<name>)) does kinda what I need but it also inserts a \x00 byte which I do NOT need

[pdurrant]

Originally Posted by At_Libitum

Hi,

well, maybe not exactly unicode, more like accented characters, so utf-8 at a minimum

I'm hitting the a wall, I need to find a function that converts this

Götterdämmerung

into this

G\xf6tterd\xe4mmerung

I've been all over the source but cannot find how it's done in python....

0xF6 for ö and 0xE4 for ä looks like iso-8859-1 (ISO Latin-1). In which case, going from a unicode string, you want

.encode("iso-8859-1")