Calibre - Vertical Text and Right to Left in Japanese

[Grinchy]

Hello,

I really love Calibre, but i can't get Japanese Vertical Text and "Right to Left" working for Converting Files.

After importing an Japanese epub or mobi it even shows right in the Viewer, but after converting it (epub -> epub/azw3/mobi, doesn't matter) the Vertical Text and Right to Left are gone.

Is there a way to fix this? I really want to convert my Books to azw3, but that's impossible with Calibre right now :-(

[kovidgoyal]

https://www.mobileread.com/forums/sh...d.php?t=186697

[Grinchy]

Sorry!

Here are the missing information!

- 無職転生 - 蛇足編 - 番外編.mobi is the Original with Vertical Text and Right to Left.

- "After Converting.mobi" is the converted file with broken Vertical text and without Right to Left.

Tried it with a lot of different files, but didn't work with any.

Log:

Buch 1 von 1 (無職転生　- 蛇足編 -) konvertieren
C:\Users\paddy\OneDrive\Calibre Portable\Calibre\app\pylib.zip\dateutil\parser.py:601: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
Conversion options changed from defaults:
  personal_doc: None
  read_metadata_from_opf: u'C:\\Users\\paddy\\AppData\\Local\\Temp\\calibre_hiqgen\\kn8q10.opf'
  verbose: 2
  output_profile: 'kindle_oasis'
  input_profile: 'kindle'
Resolved conversion options
calibre version: 3.25.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0.0,
 'book_producer': None,
 'change_justification': u'original',
 'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']",
 'chapter_mark': u'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_compress': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': u'',
 'fix_indents': True,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.KindleInput object at 0x03B5CC10>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0.0,
 'linearize_tables': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'mobi_file_type': u'old',
 'mobi_ignore_margins': False,
 'mobi_keep_original_images': False,
 'mobi_toc_at_start': False,
 'no_chapters_in_toc': False,
 'no_inline_navbars': True,
 'no_inline_toc': False,
 'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x03B5CF10>,
 'page_breaks_before': u"//*[name()='h1' or name()='h2']",
 'personal_doc': None,
 'prefer_author_sort': False,
 'prefer_metadata_cover': False,
 'pretty_print': False,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': u'C:\\Users\\paddy\\AppData\\Local\\Temp\\calibre_hiqgen\\kn8q10.opf',
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': u'',
 'search_replace': '[]',
 'series': None,
 'series_index': None,
 'share_not_sync': False,
 'smarten_punctuation': False,
 'sr1_replace': None,
 'sr1_search': None,
 'sr2_replace': None,
 'sr2_search': None,
 'sr3_replace': None,
 'sr3_search': None,
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': '[]',
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: MOBI Input running
on C:\Users\paddy\AppData\Local\Temp\calibre_hiqgen\kxxppw.mobi
Found KF8 MOBI of type 'joint'
Extracting text...
Parsing all content...
Parsing text/part0009.html ...
Parsing text/part0010.html ...
Parsing text/part0029.html ...
Parsing text/part0011.html ...
Parsing text/part0012.html ...
Parsing text/part0013.html ...
Parsing styles/0001.css ...
Parsing text/part0014.html ...
Parsing text/part0002.html ...
Parsing text/part0025.html ...
Parsing text/part0015.html ...
Parsing text/part0000.html ...
Parsing text/part0027.html ...
Parsing text/part0016.html ...
Parsing text/part0028.html ...
Parsing text/part0023.html ...
Parsing text/part0017.html ...
Parsing text/part0030.html ...
Parsing text/part0031.html ...
Parsing text/part0026.html ...
Parsing text/part0018.html ...
Parsing text/part0032.html ...
Parsing text/part0033.html ...
Parsing text/part0019.html ...
Parsing text/part0034.html ...
Parsing text/part0020.html ...
Parsing text/part0035.html ...
Parsing text/part0036.html ...
Parsing text/part0021.html ...
Parsing text/part0022.html ...
Parsing text/part0003.html ...
Parsing text/part0004.html ...
Parsing text/part0005.html ...
Parsing text/part0006.html ...
Parsing text/part0007.html ...
Parsing text/part0024.html ...
Parsing text/part0008.html ...
Parsing text/part0001.html ...
Reading TOC from NCX...
Merging user specified metadata...
Detecting structure...
Flattening CSS and remapping font sizes...
Source base font size is 16.00000pt
Removing fake margins...
Found 1 items of level: div_1
div_1  left margin stats: Counter()
div_1  right margin stats: Counter()
Cleaning up manifest...
Trimming unused files from manifest...
Creating MOBI Output...
Serializing resources...
Creating MOBI 6 output
Generating in-line TOC...
Applying case-transforming CSS...
Parsing manglecase.css ...
Parsing tocstyle.css ...
Rasterizing SVG images...
Converting XHTML to Mobipocket markup...
Serializing markup content...
  Compressing markup content...
Generating MOBI index for a book
MOBI output written to C:\Users\paddy\AppData\Local\Temp\calibre_hiqgen\adwfuw.mobi

[kovidgoyal]

Looks correct to me after converting to azw3 (note that you have to convert to azw3 or epub)

See attached, I cant read a japanese, so I cant be sure but from staring at the characters, they look correct and are still rendered vertically in the calibre viewer.

[elipsett]

I am also very interested in this problem. Let me provide a little more information using KindlePreview (latest version) on MacOS10.12, native Japanese mode.

In the two files the OP sent, the first file displays Japanese in vertical format, left-to-right, and the page advance button is (properly) on the left side of the screen.
In the second (converted) file, the Japanese is displayed horizontally, and the page advance button is on the right side of the screen (in other words, just like English text).

If this moves to a new thread it should include the sample files.

[jhowell]

Amazon's KF8 (azw3) format supports vertical text, but the old MOBI-7 format does not.

The first attachment (無職転生 - 蛇足編 - 番外編.mobi) is a Master MOBI which contains both MOBI-7 and KF8 in a single file. This type of file is produced by Amazon's Kindle Previewer or kindlegen. The reason that it works is that devices capable of displaying KF8 make use of that part of the file and ignore the MOBI-7 part.

The second attachment (After Comverting.mobi) is plain MOBI-7 format, without KF8. That is why it is so much smaller and why it does not support vertical text.

As Kovid stated, you need to convert to a format that supports vertical text, such as EPUB or KF8 (azw3) for vertical text to work properly.

[markvdvelde]

Originally Posted by jhowell

As Kovid stated, you need to convert to a format that supports vertical text, such as EPUB or KF8 (azw3) for vertical text to work properly.

I tried this too, but it failed. The original file was a vertically written azw3, but without a cover. With Calibre I added a cover, converted the file from azw3 to azw3 (and to ePub), but the text is always converted to horizontal.

Plus: mobi seems to be capable of displaying vertical text (you will see this when you open the mobi attachment on a Kindle or in the Kindle Previewer). But once I change the cover and covert it, the vertical style is gone...

[jhowell]

Originally Posted by markvdvelde

I tried this too, but it failed. The original file was a vertically written azw3, but without a cover. With Calibre I added a cover, converted the file from azw3 to azw3 (and to ePub), but the text is always converted to horizontal.

Plus: mobi seems to be capable of displaying vertical text (you will see this when you open the mobi attachment on a Kindle or in the Kindle Previewer). But once I change the cover and covert it, the vertical style is gone...

The attachment in your most recent post is a Master Mobi containing both Mobi7 and KF8 formats components in a single file. Even though it has a .mobi extension it is not pure MOBI format. It will display vertically on devices that support KF8 format.

As a test I used kindelunpack to split that file into separate .azw3 and .mobi files. As expected the split .azw3 displayed vertically and the .mobi did not.

I then added the .azw3 file to calibre, added a cover, and converted .azw3 to .azw3. The result was a .azw3 file containing vertical layout and a cover. I tested it in Kindle Previewer, Kindle for PC and a Kindle Oasis. It displayed correctly (vertically with cover) on all of these.

[markvdvelde]

Originally Posted by jhowell

As expected the split .azw3 displayed vertically and the .mobi did not.

Interesting, I had no idea that a file with a mobi extension could contain different formats!

Still, I'm having some issues with the conversion from epub to azw3, though not always. I'm not sure where the problem lies, but I'll keep tinkering with the files.

[elipsett]

Originally Posted by jhowell

As a test I used kindelunpack to split that file into separate .azw3 and .mobi files. As expected the split .azw3 displayed vertically and the .mobi did not.

Interesting, thank you.
Presumably the relevant code is in the OPF and CSS files, and not the text content files.
Now that I understand where the problem is it should be possible to find a way to fix it.

[markvdvelde]

Originally Posted by elipsett

Now that I understand where the problem is it should be possible to find a way to fix it.

If you do, please post it here! Thanks.

[elipsett]

I had a chance to do a little fiddling around.
The vast majority of the code is identical, but there are a few differences (if it's not shown below, it's the same as standard epub or Mobi):

CONTENT.OPF
In metadata:
<meta name="primary-writing-mode" content="vertical-rl" />

In spine:
<spine page-progression-direction="rtl" toc="ncx">
<itemref idref="x_p-cover" linear="no" properties="page-spread-left"/>
(and obviously more lines for the other XHTML files)

CSS file:
html,
.hltr {
-webkit-writing-mode: horizontal-tb;
-webkit-writing-mode: horizontal-tb;
}
.vrtl {
-webkit-writing-mode: vertical-rl;
-webkit-writing-mode: vertical-rl;
}


And that's it!
Everything else is pretty obvious, and applies to both horizontal and vertical display, as far as I can see.
However, there are hundreds of entries for handling gaiji, grafix, lines etc vertically. Since they are unique to vertical text, I assume the lines above are really the only ones that matter. Font definitions and body text <p> definitions are simple, and essentially identical to horizontal layouts.

Next time I have some spare time I'm going to take a random chunk of Japanese text and build an ebook around it using the above code, just to see what happens.
I'll post the results here.

The Japanese ebooks I dissected don't show the generator in the metadata list.

[elipsett]

Forgot to mention.
The simple test file used above shows the following in the CSS file, and also works just fine. It's essentially the same as the publisher-generated retail versions I checked.
body {
margin-left: 1em;
margin-right: 1em;
writing-mode:tb-rl;
-webkit-writing-mode: vertical-rl;
-webkit-writing-mode: vertical-rl; (any reason why this line is here twice?)
line-break: normal;
-epub-line-break: normal;
-webkit-line-break: normal;
}

[Envek]

@elipsett, thank you very much for provided snippets!

I added following code to page_styles.css of several ePubs converted from azw3 and now they are vertical!

.hltr {
    writing-mode: horizontal-tb;
    -webkit-writing-mode: horizontal-tb;
    -webkit-writing-mode: horizontal-tb;
}
.vrtl {
    writing-mode: vertical-rl;
    -webkit-writing-mode: vertical-rl;
    -webkit-writing-mode: vertical-rl;
}

I wish if Calibre would start adding this snippet to all converted ePubs by default.

[elipsett]

Hopefully Kovid will see this thread.
I don't have his email handy; if anyone knows it, maybe drop him a line to catch his attention?