AttributeError and Lots of crashes...

[MikesBooks]

Calibre 8.4 keeps crashing when adding books, sometimes it just won't open.

Here are some of the different errors:

Spoiler:

ERROR: Unhandled exception: <b>AttributeError</b>:'int' object has no attribute 'get'
----
Traceback (most recent call last):
File "calibre\gui2\add.py", line 516, in process_duplicate
StopIteration
---
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\gui2\tag_browser\view.py", line 1548, in recount_with_position_based_index
File "calibre\gui2\tag_browser\view.py", line 1569, in recount
File "calibre\gui2\tag_browser\model.py", line 594, in rebuild_node_tree
File "calibre\gui2\tag_browser\model.py", line 607, in _run_rebuild
File "calibre\gui2\tag_browser\model.py", line 682, in _rebuild_node_tree
File "calibre\gui2\tag_browser\model.py", line 963, in _create_node_tree
File "calibre\gui2\tag_browser\model.py", line 951, in process_one_node
AttributeError: 'int' object has no attribute 'get'


--------------
Added Those Who Stayed: A Vietnam Diary to db in: 1.1
send() failed in worker, terminating worker
Traceback (most recent call last):
File "calibre\utils\ipc\pool.py", line 378, in worker_main
File "calibre\utils\ipc\__init__.py", line 20, in eintr_retry_call
File "multiprocessing\connection.py", line 200, in send_bytes
File "multiprocessing\connection.py", line 289, in _send_bytes
BrokenPipeError: [WinError 232] The pipe is being closed
send() failed in worker, terminating worker
Traceback (most recent call last):
File "calibre\utils\ipc\pool.py", line 378, in worker_main
File "calibre\utils\ipc\__init__.py", line 20, in eintr_retry_call
File "multiprocessing\connection.py", line 200, in send_bytes
File "multiprocessing\connection.py", line 289, in _send_bytes
BrokenPipeError: [WinError 232] The pipe is being closed
----------------
File "calibre\db\fields.py", line 604, in book_value_map
File "calibre\db\fields.py", line 604, in <dictcomp>
TypeError: 'tuple_iterator' object does not support the context manager protocol
-----------------
File "calibre\utils\icu.py", line 188, in change_case
TypeError: 'type' object cannot be interpreted as an integer
----------------
File "calibre\gui2\tag_browser\model.py", line 947, in process_one_node
UnboundLocalError: cannot access local variable 'ct' where it is not associated with a value
-----------------
File "calibre\gui2\tag_browser\model.py", line 951, in process_one_node
TypeError: 'dict' object is not callable
-----------------
File "calibre\gui2\tag_browser\model.py", line 951, in process_one_node
TypeError: 'bool' object does not support the context manager protocol
------------------
File "calibre\gui2\tag_browser\model.py", line 909, in <dictcomp>
AttributeError: 'list_iterator' object has no attribute 'tag'
-------------------
File "calibre\gui2\tag_browser\model.py", line 951, in process_one_node
AttributeError: 'int' object has no attribute 'get'
-----------------
File "calibre\db\tables.py", line 654, in <dictcomp>
TypeError: unhashable type: 'dict'

Things I've tried:
-Full text - Disabiled and Deleted the full text search db files

-Metadata - Deleted and spent 17 hrs rebuilding the metadata.db file

-Maintenance - Check library showed no errors

-Uninstalled & Reinstalled - 8.4 and even downgraded to a version of 7 I still had

-Windows - SFC /scan now and dism /online /cleanup-image /scanhealth and restorehealth show no problems

-Locations - Database is on an internal D drive, and I have tried moving calibre program from C drive to D drive and back again with no change.

- Debug & No 3rd Party Plugins: still crashes in debug mode and debug w/no 3rd party plugins

There are 27,173 books in the database

I've gone through the threads trying a bunch of different things and I am stumped

[MikesBooks]

On a random thought, I created a new library and loaded 8.4.1000 and so far no crashes or issues.

Going back to the other library and still all the crashes in the world.

[BetterRed]

Just to make sure it is not interference from a third party program, try Windows Safe mode.

You might have to rebuild the library, see ==>> https://www.mobileread.com/forums/sh...93&postcount=4

BR

[MikesBooks]

Originally Posted by BetterRed

Just to make sure it is not interference from a third party program, try Windows Safe mode.

You might have to rebuild the library, see ==>> https://www.mobileread.com/forums/sh...93&postcount=4

BR

Thanks for the suggestion!

I had tried it in windows safe mode, and I doubly checked to make sure even windows security and defender were set to off.

I have a feeling I might just have to rebuild the library though ... I tried rebuilding the metadata.db and that didn't solve it either.

It's a pretty new computer, less than a year old, and the only things installed at Calibre, Office 2017, and Adobe Digital Editions, Kindle 4 PC, and Firefox.

I really hope I don't have to re-add all 27k books... but I might just have to at this point. Oof!

[kovidgoyal]

Those definitely look like corruption issues in your library. You say you tried rebuild library database? What happened when you did?

[MikesBooks]

Originally Posted by kovidgoyal

Those definitely look like corruption issues in your library. You say you tried rebuild library database? What happened when you did?

Moderator Notice
Changed to attachment. Please follow MR image guidelines

I did, and after over 12 hrs or so it showed it as done. So I added another book and it immediately crashed and gave me more of those errors.

I did both of the delete metadata and restore, and then I tried the starting from a blank and add books from the old folder.

With the delete metadata and restore, I still get crashes. With the add books it added only 900 of the 27k and then crashed.

I also checked both hard drives (internal C and D) for errors (chkdsk and windows error checking) and defrag and found no problems. Even downloaded a hard drive S.M.A.R.T program and it found no issues with either drive.

see attachment

But it's still crashing constantly and I can't figure it out.

[BetterRed]

Originally Posted by MikesBooks

I did both of the delete metadata and restore, and then I tried the starting from a blank and add books from the old folder.

With the delete metadata and restore, I still get crashes. With the add books it added only 900 of the 27k and then crashed.

The first screen shot you posted is from the Restore database feature, if you add books from an existing library folder, as per the link to Kovid's recommendation in my previous post, you will see something like this:



BR

[MikesBooks]

Originally Posted by BetterRed

The first screen shot you posted is from the Restore database feature, if you add books from an existing library folder, as per the link to Kovid's recommendation in my previous post, you will see something like this:

Attachment 215817

BR

Yeah I tried that and it got to 900 books and then just crashed out. No screen shot because it was going good and then... was gone!

[kovidgoyal]

Reboot in safe mode and run

calibredb restore_database -r

in a windows command prompt

After that finishes try starting calibre while still in safe mode as

calibre --ignore-plugins

and see if it is working.

[MikesBooks]

Some success! In safe mode it took just over 24hrs, but it rebuilt and seemed to run fine. Only thing missing were my custom icons and custom metadata. I really hope I didn't lose those.

The calibre_db_restore_report.txt does show a few errors like this:

Spoiler:

D:\Full Library-2017-07-25\Alexander Mladenov\Sukhoi Su-25 Frogfoot (10759) with error:
Traceback (most recent call last):
File "calibre\db\restore.py", line 206, in scan_library
File "calibre\db\restore.py", line 231, in process_dir
File "calibre\db\restore.py", line 35, in read_opf
File "calibre\ebooks\metadata\opf2.py", line 642, in __init__
File "calibre\ebooks\metadata\utils.py", line 47, in parse_opf
File "calibre\utils\xml_parse.py", line 35, in safe_xml_fromstring
File "src/lxml/etree.pyx", line 3287, in lxml.etree.fromstring
File "src/lxml/parser.pxi", line 1989, in lxml.etree._parseMemoryDocument
File "src/lxml/parser.pxi", line 1876, in lxml.etree._parseDoc
File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc
File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError
File "<string>", line 1
lxml.etree.XMLSyntaxError: StartTag: invalid element name, line 1, column 2

and

Spoiler:

Traceback (most recent call last):
File "calibre\db\restore.py", line 302, in restore_books
File "calibre\db\cache.py", line 85, in call_func_with_lock
File "calibre\db\cache.py", line 2938, in restore_book
File "calibre\db\cache.py", line 2156, in create_book_entry
File "calibre\db\backend.py", line 1171, in execute
File "C:\t\t\apsw-42s16m2x\src\cursor.c", line 240, in resetcursor
apsw.ConstraintError: ConstraintError: UNIQUE constraint failed: books.id

But it was just like 10 books so not a big deal.

When restarting the computer in normal mode I first got this error:

Spoiler:

calibre, version 8.4.100
ERROR: Bad database location: Bad database location 'D:\\Full Library-2017-07-25'. Will start with a new, empty calibre library

Traceback (most recent call last):
File "calibre\gui2\main.py", line 323, in initialize_db
File "calibre\db\legacy.py", line 192, in __init__
File "calibre\db\legacy.py", line 46, in create_backend
File "calibre\db\backend.py", line 551, in __init__
File "calibre\db\backend.py", line 990, in initialize_notes
File "calibre\db\notes\connect.py", line 76, in __init__
File "os.py", line 225, in makedirs
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'D:\\Full Library-2017-07-25\\.calnotes\\resources'

I changed it to run as administrator and it opened fine the first couple of times, and then it crashed hard the next few. Task manager shows nothing running in the background either, so it's a hard closure.

When I attempted to add 3 books (in admin mode), once again, hard closure, and I got that database error once. I retried and now I'm getting the original error when adding books:

Spoiler:

Traceback (most recent call last):
File "calibre\gui2\tag_browser\model.py", line 1389, in _get_category_nodes
File "calibre\db\cache.py", line 1555, in get_categories
File "calibre\db\categories.py", line 252, in get_categories
File "calibre\db\fields.py", line 183, in get_categories
File "calibre\db\fields.py", line 184, in <genexpr>
TypeError: '>' not supported between instances of 'generator' and 'int'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\gui2\tag_browser\view.py", line 1569, in recount
File "calibre\gui2\tag_browser\model.py", line 594, in rebuild_node_tree
File "calibre\gui2\tag_browser\model.py", line 607, in _run_rebuild
File "calibre\gui2\tag_browser\model.py", line 612, in _rebuild_node_tree
File "calibre\gui2\tag_browser\model.py", line 1396, in _get_category_nodes
File "calibre\db\cache.py", line 1555, in get_categories
File "calibre\db\categories.py", line 217, in get_categories
File "calibre\db\fields.py", line 604, in book_value_map
File "calibre\db\fields.py", line 604, in <dictcomp>
TypeError: unhashable type: 'dict'

It seemed to work fine under safe mode but still being problematic on regular.

This is a pretty new, clean build of windows, no a/v software like malwarebytes, it's on an internal hard drive and has been working fine since 2017!

Is there anything else I can try?

Thanks for the help!