Converting FB2->EPUB, TOC is only half the size

[whiteowl]

I have a huge book in FB2 - 49521 Kb, 2262 chapters.
Tried to read it as is in a Viewer - TOC shows up to chapter 1043.
So I converted the book into EPUB with built-in converter.

Looking at the contents of EPUB I see all chapters correctly converted (several chapters per `index_split_N.xhtml` with last file `index_split_1205.xhtml`. And inside it, I see the final chapter up to the "Fin". So far so good...

But the issue is with `toc.ncx` - it stops in the middle.

      <navPoint id="uYHYxzqrSIyP8DRv35YFT8D" playOrder="1050">
        <navLabel>
          <text>Chapter 1043. 12 people</text>
        </navLabel>
        <content src="index_split_1049.xhtml"/>
      </navPoint>

Bug in the reader part of FB2?

[kovidgoyal]

https://www.mobileread.com/forums/sh...d.php?t=186697

[whiteowl]

I am not sure I can send you the original file - it is almost 50 MEGA bytes.

But in the conversion log, I see

Convert book 1 of 1 (Martial World)
Conversion options changed from defaults:
  read_metadata_from_opf: 'C:\\Users\\George\\AppData\\Local\\Temp\\calibre-gd7a55ez\\ayrh675q.opf'
  verbose: 2
  cover: 'C:\\Users\\George\\AppData\\Local\\Temp\\calibre-gd7a55ez\\4_qwrr4c.jpeg'
  output_profile: 'kindle_pw3'
Resolved conversion options
calibre version: 9.8.0
{'add_alt_text_to_img': False,
 'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0.0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': "//*[((name()='h1' or name()='h2') and re:test(., "
            "'\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', "
            "'i')) or @class = 'chapter']",
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': 'C:\\Users\\George\\AppData\\Local\\Temp\\calibre-gd7a55ez\\4_qwrr4c.jpeg',
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_split_on_page_breaks': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_max_image_size': 'none',
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': '',
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000020BAD5B9A90>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': '//h:h1',
 'level2_toc': '//h:h2',
 'level3_toc': '//h:h3',
 'line_height': 0.0,
 'linearize_tables': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_fb2_toc': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.KindlePaperWhite3Output object at 0x0000020BAD5C4590>,
 'page_breaks_before': "//*[name()='h1' or name()='h2']",
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': 'C:\\Users\\George\\AppData\\Local\\Temp\\calibre-gd7a55ez\\ayrh675q.opf',
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': '[]',
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': None,
 'sr1_search': None,
 'sr2_replace': None,
 'sr2_search': None,
 'sr3_replace': None,
 'sr3_search': None,
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': '[]',
 'transform_html_rules': '[]',
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: FB2 Input running
on C:\Users\George\AppData\Local\Temp\calibre-gd7a55ez\vsq8fb_v.fb2
Parsing XML...
Converting XML to HTML...
Parsing all content...
Parsing index.xhtml ...
Forcing index.xhtml into XHTML namespace
Parsing inline-styles.css ...
Generating default TOC from spine...
Merging user specified metadata...
Detecting structure...
Auto generated TOC with 1050 entries.
Flattening CSS and remapping font sizes...
Source base font size is 12.00000pt
Removing fake margins...
Found 2 items of level: div_1
Found 94103 items of level: div_2
Found 2 items of level: div_3
Found 65783 items of level: p_1
Ignoring level div_3
div_1  left margin stats: Counter({'': 1})
div_1  right margin stats: Counter({'': 1})
div_2  left margin stats: Counter()
div_2  right margin stats: Counter()
p_1  left margin stats: Counter({'0': 65783})
p_1  right margin stats: Counter({'0': 65783})
Cleaning up manifest...
Trimming unused files from manifest...
Trimming 'poster.jpg.jpg' from manifest
Creating EPUB Output...
Splitting markup on page breaks and flow limits, if any...
		Splitting on page-break at id=calibre_toc_1
Adjusted split point to ancestor
		Splitting on page-break at id=calibre_toc_2
....
		Splitting on page-break at id=calibre_toc_1049
		Splitting on page-break at id=calibre_toc_1050
	Looking for large trees in index.xhtml...
	Found large tree #1049
		Splitting...
			Split point: {http://www.w3.org/1999/xhtml}h4 /*/*[2]/*[54]/*[55851]
			Split tree still too large: 16114.1943359375 KB
		Splitting...
			Split point: {http://www.w3.org/1999/xhtml}h4 /*/*[2]/*[54]/*[30555]
			Split tree still too large: 8571.6767578125 KB
		Splitting...
			Split point: {http://www.w3.org/1999/xhtml}h4 /*/*[2]/*[54]/*[16166]
			Split tree still too large: 4346.8828125 KB
.....
		Splitting...
			Split point: {http://www.w3.org/1999/xhtml}h4 /*/*[2]/*/*[722]
			Committed sub-tree #154 (211.3525390625 KB)
			Committed sub-tree #155 (207.0234375 KB)
			Split tree still too large: 339.84765625 KB
		Splitting...
			Split point: {http://www.w3.org/1999/xhtml}h4 /*/*[2]/*/*[555]
			Committed sub-tree #156 (171.1171875 KB)
			Committed sub-tree #157 (169.1904296875 KB)
	Split into 1206 parts
Removing anchor from TOC href: index_split_000.xhtml#calibre_toc_1
Removing anchor from TOC href: index_split_001.xhtml#calibre_toc_2
.....
Removing anchor from TOC href: index_split_1048.xhtml#calibre_toc_1049
Removing anchor from TOC href: index_split_1049.xhtml#calibre_toc_1050
EPUB output written to C:\Users\George\AppData\Local\Temp\calibre-gd7a55ez\e40a3m_e.epub

No changes of options from default. Just a standard `FB2->EPUB` conversion.

The final file is smaller (just 14Mb), but still too large for sharing. It contains complete text from the original. Only problem is with table of contents.

[Jaws]

Whiteowl, does the FB2 file include page breaks or page-break markers attempting to preserve the pagination of an original printed edition? Even one error or inconsistency somewhere can blow up read of the table of contents — especially if it's in the table of contents file itself.

And is this large FB2 file a single publication, or is it a merging of several publications?

[whiteowl]

Originally Posted by Jaws

Whiteowl, does the FB2 file include page breaks or page-break markers attempting to preserve the pagination of an original printed edition? Even one error or inconsistency somewhere can blow up read of the table of contents — especially if it's in the table of contents file itself.

No, nothing like that. Checked "the border" chapter - just a normal text.

Originally Posted by Jaws

And is this large FB2 file a single publication, or is it a merging of several publications?

Just a single book.

If you look at log, there is a message in preparation phase:

Auto generated TOC with 1050 entries.

So I am guessing, once the preparation guessed it would be 1050 chapters, it is not adjusted during the actual conversion of chapters.