Error creating library catalog

RSG2 · 12-08-2021, 01:24 AM

I am having the worst luck trying to catalog my library; it worked long (eons) ago, but these days I keep getting the following error:

Generate catalog
DeACSM v0.0.12: Copying needed library files from plugin zip
Integration status: True
Initialized urlfixer
Sections: Authors, Titles, Series, Genres, Recently Added, Descriptions
Catalog_EPUB_MOBI('Calibre Library'): Generating azw3 for kobo in GUI environment, locale: 'English'
connected_device: 'None'
book count: 12555
opts:
author_clip: 100
catalog_title: 'My Books AZW'
connected_kindle: False
creator: 'Tuesday, December 7, 2021'
cross_reference_authors: True
description_clip: 380
exclude_genre: '\\[.+\\]|^\\+$'
exclusion_rules: (('Catalogs', 'Tags', 'Catalog'),)
fmt: 'azw3'
genre_source_field: 'Tags'
header_note_source_field: ''
merge_comments_rule: '::True'
output_profile: 'kobo'
prefix_rules: ()
search_text: None
sort_by: None
sort_descriptions_by_author: True
sync: False
thumb_width: '1.00'
use_existing_cover: False
Begin catalog source generation (0:00:00)

unhandled exception in catalog generator
Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 81, in gui_catalog
File "calibre\library\catalogs\epub_mobi.py", line 406, in run
File "calibre\library\catalogs\epub_mobi_builder.py ", line 330, in build_sources
File "calibre\library\catalogs\epub_mobi_builder.py ", line 2189, in generate_html_by_genres
File "calibre\library\catalogs\epub_mobi_builder.py ", line 2296, in generate_html_by_genre
File "calibre\library\catalogs\epub_mobi_builder.py ", line 1382, in format_prefix
File "calibre\ebooks\BeautifulSoup.py", line 34, in BeautifulSoup
File "calibre\ebooks\BeautifulSoup.py", line 23, in parse_html
File "html5_parser\soup.py", line 130, in parse
ModuleNotFoundError: No module named 'html5_parser.book_3264'

The book number will change between runs (I presume this is the number that is attached to the book in the library files, correct?); I am running the latest version of Calibre on Windows 10, with 16 Gigs of RAM available, and approximately 12.5K books to catalog. Does anyone recognize this problem, and have a solution? Thanks in advance.

kovidgoyal · 12-08-2021, 10:06 AM

That's a pretty amazing error, offhand I dont know how its even possible without some kind of extensive memory corruption. First thing to so is

Uninstall, delete C:\\Program Files\\Calibre2
(or wherever you chose to install calibre). Then re-install. And
dont worry, uninstalling will not affect your books/settings.

RSG2 · 12-08-2021, 08:27 PM

Quote:

Originally Posted by kovidgoyal

That's a pretty amazing error, offhand I dont know how its even possible without some kind of extensive memory corruption. First thing to so is

Uninstall, delete C:\\Program Files\\Calibre2
(or wherever you chose to install calibre). Then re-install. And
dont worry, uninstalling will not affect your books/settings.

Done, with no plugins yet. Stopped again, but this time with no actual error message. I'll try it again and report back.

kovidgoyal · 12-08-2021, 08:55 PM

Then attach the metadata.db file from your calibre library folder and I will see if I can reproduce.

RSG2 · 12-09-2021, 12:44 AM

413 Request Entity Too Large

That's what I get when I try to attach it here. It is 114,712KB...

kovidgoyal · 12-09-2021, 12:51 AM

open a bug report and attach it there https://calibre-ebook.com/bugs

RSG2 · 12-09-2021, 01:08 AM

https://we.tl/t-zLBfUFYO0S

kovidgoyal · 12-09-2021, 03:16 AM

Yeah there is definitely memory corruption going on with that, though tracking down the cause is going to be difficult given such a large db.

RSG2 · 12-09-2021, 06:08 PM

What do I need to do? What kind of memory corruption is going on?

kovidgoyal · 12-09-2021, 09:23 PM

If you can come up with a smaller data set that causes the issue it would help. Otherwise I will investigate when I have some time. Sadly the catalog code is largely un-maintained as its original developer GRiker left a few years ago.

12-08-2021, 01:24 AM	#1
RSG2 Junior Member Posts: 5 Karma: 10 Join Date: Dec 2021 Device: None	Error creating library catalog I am having the worst luck trying to catalog my library; it worked long (eons) ago, but these days I keep getting the following error: Generate catalog DeACSM v0.0.12: Copying needed library files from plugin zip Integration status: True Initialized urlfixer Sections: Authors, Titles, Series, Genres, Recently Added, Descriptions Catalog_EPUB_MOBI('Calibre Library'): Generating azw3 for kobo in GUI environment, locale: 'English' connected_device: 'None' book count: 12555 opts: author_clip: 100 catalog_title: 'My Books AZW' connected_kindle: False creator: 'Tuesday, December 7, 2021' cross_reference_authors: True description_clip: 380 exclude_genre: '\\[.+\\]\|^\\+$' exclusion_rules: (('Catalogs', 'Tags', 'Catalog'),) fmt: 'azw3' genre_source_field: 'Tags' header_note_source_field: '' merge_comments_rule: '::True' output_profile: 'kobo' prefix_rules: () search_text: None sort_by: None sort_descriptions_by_author: True sync: False thumb_width: '1.00' use_existing_cover: False Begin catalog source generation (0:00:00) unhandled exception in catalog generator Traceback (most recent call last): File "runpy.py", line 194, in _run_module_as_main File "runpy.py", line 87, in _run_code File "site.py", line 82, in <module> File "site.py", line 77, in main File "site.py", line 49, in run_entry_point File "calibre\utils\ipc\worker.py", line 216, in main File "calibre\gui2\convert\gui_conversion.py", line 81, in gui_catalog File "calibre\library\catalogs\epub_mobi.py", line 406, in run File "calibre\library\catalogs\epub_mobi_builder.py ", line 330, in build_sources File "calibre\library\catalogs\epub_mobi_builder.py ", line 2189, in generate_html_by_genres File "calibre\library\catalogs\epub_mobi_builder.py ", line 2296, in generate_html_by_genre File "calibre\library\catalogs\epub_mobi_builder.py ", line 1382, in format_prefix File "calibre\ebooks\BeautifulSoup.py", line 34, in BeautifulSoup File "calibre\ebooks\BeautifulSoup.py", line 23, in parse_html File "html5_parser\soup.py", line 130, in parse ModuleNotFoundError: No module named 'html5_parser.book_3264' The book number will change between runs (I presume this is the number that is attached to the book in the library files, correct?); I am running the latest version of Calibre on Windows 10, with 16 Gigs of RAM available, and approximately 12.5K books to catalog. Does anyone recognize this problem, and have a solution? Thanks in advance.

12-09-2021, 12:44 AM	#5
RSG2 Junior Member Posts: 5 Karma: 10 Join Date: Dec 2021 Device: None	Metadata DB, as requested 413 Request Entity Too Large That's what I get when I try to attach it here. It is 114,712KB...

12-09-2021, 01:08 AM	#7
RSG2 Junior Member Posts: 5 Karma: 10 Join Date: Dec 2021 Device: None	Database requested https://we.tl/t-zLBfUFYO0S

12-09-2021, 06:08 PM	#9
RSG2 Junior Member Posts: 5 Karma: 10 Join Date: Dec 2021 Device: None	Understood... What do I need to do? What kind of memory corruption is going on?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Creating Catalog	Tanjamuse	Calibre	6	02-28-2020 01:44 AM
Error creating catalog	quini	Library Management	5	02-09-2020 03:14 PM
Error creating an epub catalog	pscynthetiq	Library Management	2	09-30-2017 07:25 PM
Problem with creating catalog	Jade Aislin	Library Management	2	07-03-2013 05:36 PM
Conversion Error when creating Catalog	thangnh	Library Management	1	02-21-2011 08:38 AM

12-08-2021, 10:06 AM	#2
kovidgoyal creator of calibre Posts: 45,604 Karma: 28548974 Join Date: Oct 2006 Location: Mumbai, India Device: Various	That's a pretty amazing error, offhand I dont know how its even possible without some kind of extensive memory corruption. First thing to so is Uninstall, delete C:\\Program Files\\Calibre2 (or wherever you chose to install calibre). Then re-install. And dont worry, uninstalling will not affect your books/settings.

12-08-2021, 08:55 PM	#4
kovidgoyal creator of calibre Posts: 45,604 Karma: 28548974 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Then attach the metadata.db file from your calibre library folder and I will see if I can reproduce.

12-09-2021, 12:51 AM	#6
kovidgoyal creator of calibre Posts: 45,604 Karma: 28548974 Join Date: Oct 2006 Location: Mumbai, India Device: Various	open a bug report and attach it there https://calibre-ebook.com/bugs

12-09-2021, 03:16 AM	#8
kovidgoyal creator of calibre Posts: 45,604 Karma: 28548974 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Yeah there is definitely memory corruption going on with that, though tracking down the cause is going to be difficult given such a large db.

12-09-2021, 09:23 PM	#10
kovidgoyal creator of calibre Posts: 45,604 Karma: 28548974 Join Date: Oct 2006 Location: Mumbai, India Device: Various	If you can come up with a smaller data set that causes the issue it would help. Otherwise I will investigate when I have some time. Sadly the catalog code is largely un-maintained as its original developer GRiker left a few years ago.