Extended ASCII characters in txt file

pshute · 02-26-2012, 09:08 PM

I have a txt file I want to convert to ePub format for my Sony PRS-T1, but it contains many instances of extended ASCII line-drawing characters. These show up on the reader as various letters with accents, etc, instead of lines. (E.g. ASCII value C4 hex should be a horizontal line.)

I assume this means the reader can't handle extended ASCII characters.

Can someone give me some advice about what to do to fix it? Is there a way to make the device display them properly? Or should I be doing a search and replace to change them to something else? If the latter, what should I change them to ?

theducks · 02-26-2012, 10:34 PM

Quote:

Originally Posted by pshute

I have a txt file I want to convert to ePub format for my Sony PRS-T1, but it contains many instances of extended ASCII line-drawing characters. These show up on the reader as various letters with accents, etc, instead of lines. (E.g. ASCII value C4 hex should be a horizontal line.)

I assume this means the reader can't handle extended ASCII characters.

Can someone give me some advice about what to do to fix it? Is there a way to make the device display them properly? Or should I be doing a search and replace to change them to something else? If the latter, what should I change them to ?

The keyword is Codepage (it has an optional setting on the conversions form)
you need to select the encoding used: ie. CP1252

pshute · 02-26-2012, 11:25 PM

Quote:

Originally Posted by theducks

The keyword is Codepage (it has an optional setting on the conversions form)
you need to select the encoding used: ie. CP1252

Thanks for that. But which codepage should I pick?

I've tried CP1252, ascii, Latin 1, utf-8. Only ascii made it look any different, boxes for every line drawing character.

I also don't understand why choosing ascii made the conversion take several minutes compared to about half a minute for the others.

pshute · 02-27-2012, 05:55 AM

Quote:

Originally Posted by pshute

Thanks for that. But which codepage should I pick?

I've tried CP1252, ascii, Latin 1, utf-8. Only ascii made it look any different, boxes for every line drawing character.

I tried them all, and none worked. then I realised I can type enything in the character encoding box, so I entered CP437, which Wikipedia indicated was the right one.

That worked (in the Calibre reader, havent tried on the Sony yet), but now I see the columns are misaligned. It looks like the whitespace has been collapsed.

I tried ticking Preserve Whitespace, but it didn't help. Now what?

I just tried it on the device, and now the box drawing characters are all question marks. Maybe I should be replacing the characters instead?

Anyone know if a Sony can do box drawing characters?

itimpi · 02-27-2012, 07:41 AM

If you want ascii art to stay aligned then you will need to make sure that a mono-space type font is selected.

Question marks on the device tends to mean that the font selected does not have relevant the characters in it.

Rob Lister · 02-27-2012, 08:25 AM

Quote:

Originally Posted by pshute

I tried them all, and none worked. then I realised I can type enything in the character encoding box, so I entered CP437, which Wikipedia indicated was the right one.

That worked (in the Calibre reader, havent tried on the Sony yet), but now I see the columns are misaligned. It looks like the whitespace has been collapsed.

I tried ticking Preserve Whitespace, but it didn't help. Now what?

I just tried it on the device, and now the box drawing characters are all question marks. Maybe I should be replacing the characters instead?

Anyone know if a Sony can do box drawing characters?

Can you show us the resulting css and xhtml?

theducks · 02-27-2012, 09:17 AM

Quote:

Originally Posted by pshute

I tried them all, and none worked. then I realised I can type enything in the character encoding box, so I entered CP437, which Wikipedia indicated was the right one.

That worked (in the Calibre reader, havent tried on the Sony yet), but now I see the columns are misaligned. It looks like the whitespace has been collapsed.

I tried ticking Preserve Whitespace, but it didn't help. Now what?

I just tried it on the device, and now the box drawing characters are all question marks. Maybe I should be replacing the characters instead?

Anyone know if a Sony can do box drawing characters?

Also remember HTML tends to ignore multiple whitespace

If you wrap the Ascii Art in <pre> tags, that should stop that part of the problem.

You may need to embed (and specify) a font that has the old upper ASCII chars if your device does not have one for you to specify.

pshute · 02-27-2012, 03:53 PM

Quote:

Originally Posted by Rob Lister

Can you show us the resulting css and xhtml?

How do I get to it, please?

pshute · 02-27-2012, 04:00 PM

Quote:

Originally Posted by theducks

Also remember HTML tends to ignore multiple whitespace

If you wrap the Ascii Art in <pre> tags, that should stop that part of the problem.

You may need to embed (and specify) a font that has the old upper ASCII chars if your device does not have one for you to specify.

I just found a list of supplied fonts at:
https://wiki.mobileread.com/wiki/PRST...or_xhtml_files
and a method of adding more at:
http://blog.the-ebook-reader.com/201...t1-no-rooting/

Whether any of the supplied fonts support these characters, I think I'll have to work out by trial and error. It'll definitely have to be a monospaced font, or there's no point to it.

Can I just wrap the whole document in pre tags? I dread having to work out some regex to detect lines with ascii art in them. I can't just edit the file manually because it's regularly updated and I'll just end up having to do it again and again.

Rob Lister · 02-28-2012, 06:35 AM

Quote:

Originally Posted by pshute

How do I get to it, please?

If you don't have an epub editor, the easist way is to rename the file extension of the book to .zip, like ...

mybook.epub

to

mybook.zip

Your computer will 'warn' you, but do it anyway. It will be ok.

the book will turn into a folder.

Open the folder and there are the folders and .xhtml files that make up your book.

Hunt around till you find the section that contains the offending text.

When you're done, rename it again to .epub.

that's it.

OR!!!

Download and install an epub editor.

Sigal is what I use (I don't think a better [s]free[/s] one exists). Search the forums for a download link ... or google it.

frostschutz · 02-28-2012, 06:57 AM

If possible try to replace ascii art with sensible HTML such as <table> or <hr>.

Otherwise you'll just always run into problems with that document. Even if you find a font that displays the characters, it will only display fine as long as you also choose a font size small enough to fit the width of the ascii art on the screen. Which could be a rather small font.

02-26-2012, 09:08 PM	#1
pshute Enthusiast Posts: 39 Karma: 189896 Join Date: Feb 2012 Device: Sony PRS-T1, iPad Air	Extended ASCII characters in txt file I have a txt file I want to convert to ePub format for my Sony PRS-T1, but it contains many instances of extended ASCII line-drawing characters. These show up on the reader as various letters with accents, etc, instead of lines. (E.g. ASCII value C4 hex should be a horizontal line.) I assume this means the reader can't handle extended ASCII characters. Can someone give me some advice about what to do to fix it? Is there a way to make the device display them properly? Or should I be doing a search and replace to change them to something else? If the latter, what should I change them to ?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Non-ASCII characters in recipe titles show as Ã¼	bubak	Recipes	2	11-30-2011 07:49 AM
Converting non-ASCII characters	davidnye	Recipes	0	08-20-2011 07:16 PM
advanced text search and non-ascii characters	msz59	General Discussions	0	05-05-2011 09:47 AM
Typing non-ASCII characters with the keyboard	Edmundo	Amazon Kindle	5	01-20-2011 01:18 PM
Is it possible to sent books to device with filename in non-ascii characters?	flyisland	Calibre	8	10-16-2010 05:35 AM

02-27-2012, 07:41 AM	#5
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	If you want ascii art to stay aligned then you will need to make sure that a mono-space type font is selected. Question marks on the device tends to mean that the font selected does not have relevant the characters in it.

02-28-2012, 06:57 AM	#11
frostschutz Linux User Posts: 2,282 Karma: 6123806 Join Date: Sep 2010 Location: Heidelberg, Germany Device: none	If possible try to replace ascii art with sensible HTML such as <table> or <hr>. Otherwise you'll just always run into problems with that document. Even if you find a font that displays the characters, it will only display fine as long as you also choose a font size small enough to fit the width of the ascii art on the screen. Which could be a rather small font.