Crashing on launch on macOS

[tamaracks]

I was doing some stuff in calibre, and things started acting a bit weird. I was using an Action Chain to unpack some Kindle books, and it started throwing errors that made it seem it was trying to act on books other than the ones that were selected.

I figured I'd try quitting and relaunching calibre and see what happened, but now it's crashing every time I try to launch it. I tried rebooting my Mac and downloading a new version of calibre, but that didn't help.

I am attaching the crash report and also the output of

/Applications/calibre.app/Contents/MacOS/calibre-debug -g

as txt files in this post.

Any ideas what's gone wrong?

Thanks.

[KevinH]

Just a guess but based on your debug.txt:

Failed to initialize plugin: '/Users/tamara/Library/Preferences/calibre/plugins/Goodreads More Tags.zip'

So try manually removing that Goodreads plugin and any old DeDRM plugins as well.

Does it still crash at start up?

[kovidgoyal]

This is definitely plugin related, run calibre as


/Applications/calibre.app/Contents/MacOS/calibre --ignore-plugins

and it should hopefully startup fine. Although looking at the debug log some plugin has created some very strange custom column templates, which might be causing issues as well.

[tamaracks]

Originally Posted by kovidgoyal

This is definitely plugin related, run calibre as


/Applications/calibre.app/Contents/MacOS/calibre --ignore-plugins

and it should hopefully startup fine. Although looking at the debug log some plugin has created some very strange custom column templates, which might be causing issues as well.

Alas, still crashing.

I removed the goodreads more tags plugin and the dedrm plugin, and any associated files I could see. I also remove everything in the custom_recipes folder, because that was all ancient and nothing I'm using now.

Launching with the ignore plugins flag now gives this output:

❯ /Applications/calibre.app/Contents/MacOS/calibre --ignore-plugins                                                   ─╯
['Not all dates have values']
fish: Job 1, '/Applications/calibre.app/Conte…' terminated by signal SIGSEGV (Address boundary error)

New debug output and crash report attached.

Can you tell me more about what you are seeing with custom columns?

[kovidgoyal]

See

Traceback (most recent call last):
  File "calibre/utils/formatter.py", line 1722, in safe_format
  File "calibre/utils/formatter.py", line 1616, in evaluate
  File "calibre/utils/formatter.py", line 1524, in _eval_program
  File "calibre/utils/formatter.py", line 851, in program
  File "calibre/utils/formatter.py", line 865, in expression_list
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1415, in expr
  File "calibre/utils/formatter.py", line 873, in do_node_if
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1414, in expr
  File "calibre/utils/formatter.py", line 865, in expression_list
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1415, in expr
  File "calibre/utils/formatter.py", line 985, in do_node_func
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1414, in expr
  File "calibre/utils/formatter.py", line 865, in expression_list
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1415, in expr
  File "calibre/utils/formatter.py", line 1108, in do_node_field
  File "calibre/utils/formatter.py", line 1106, in do_node_field
  File "calibre/utils/formatter.py", line 829, in error
ValueError: Interpreter: Unknown field '#catalog_status' - line number 2
Error evaluating column named: *row1098663422593704131

Add

--with-library=/tmp/empty

[tamaracks]

Originally Posted by kovidgoyal

See

Traceback (most recent call last):
  File "calibre/utils/formatter.py", line 1722, in safe_format
  File "calibre/utils/formatter.py", line 1616, in evaluate
  File "calibre/utils/formatter.py", line 1524, in _eval_program
  File "calibre/utils/formatter.py", line 851, in program
  File "calibre/utils/formatter.py", line 865, in expression_list
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1415, in expr
  File "calibre/utils/formatter.py", line 873, in do_node_if
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1414, in expr
  File "calibre/utils/formatter.py", line 865, in expression_list
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1415, in expr
  File "calibre/utils/formatter.py", line 985, in do_node_func
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1414, in expr
  File "calibre/utils/formatter.py", line 865, in expression_list
  File "calibre/utils/formatter.py", line 1417, in expr
  File "calibre/utils/formatter.py", line 1415, in expr
  File "calibre/utils/formatter.py", line 1108, in do_node_field
  File "calibre/utils/formatter.py", line 1106, in do_node_field
  File "calibre/utils/formatter.py", line 829, in error
ValueError: Interpreter: Unknown field '#catalog_status' - line number 2
Error evaluating column named: *row1098663422593704131

Add

--with-library=/tmp/empty

I see what you mean, although I don't understand exactly what would be driving those messages. I have been doing a bunch of stuff with custom columns to track where I am at massaging ebook metadata and EPUB formatting. Much of it I have been filling in using Action Chains and template functions. But I'm not sure what it is that would be trying to do anything on launch?

Launching with the empty library works. Do you have a suggestion for what to do after that?

[tamaracks]

Oh, and this is what outputs in terminal when launching with the empty library:

Using library at /tmp/empty
calibre_plugins.librarything_match.fuzzywuzzy.fuzz:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
ActionChains::actions/calibre_actions2.py - exception when loading translations
ActionChains::actions/sort.py - exception when loading translations

[tamaracks]

I couldn't stop thinking about the two bits of information in the debug output that pertain to stuff I did, which was about the column #catalog_status and the message 'Not all dates have values'.

So I perused the metadata_db_prefs_backup.json file for my library. I don't have the #catalog_status column anymore, I do have one called #cat_status (I guess I decided I wanted to recreate it with a shorter name). But #catalog_status still appears in a couple places; a saved search and a column color rule.

Saved search:

Spoiler:

"Missing Formats": "formats:false and not #amzn_resource_type:\"sample\" and not #amzn_resource_type:\"borrowed\" and not #origin_type:\"borrowed\" and not #catalog_status:\"wishlist\" and not #catalog_status:\"loan-returned\" and not #catalog_status:\"loan-active\" and not #catalog_status:\"unknown\" and not #catalog_status:\"physical\"",

Column color rule:

Spoiler:

    [
      "*row",
      "program:\ntest(strcmp(field('#catalog_status'), \"Wishlist\", '', '1', ''), '#929292', '');"
    ],

The phrase 'Not all dates have values' shows up in a template function (which I am pretty sure is also the basis of a custom column).

Template function:

Spoiler:

    [
      "bookfusion_status",
      "",
      0,
      "program:\n\tepubmod = select(formats_modtimes('dd MMM yyyy hh:mm'),'EPUB');\n\tbfupload = field('#bookfusion_upload');\n\tbfyn = field('#bookfusion_upload_yn');\n\tepubyn = has_epub();\n\tdiff = compare_bf_epub_dates();\n\tnextstep = field('#next_step');\n\tnextstepbf = contains(nextstep,'.*bookfusion.*','yes',contains(nextstep,'.*done.*','yes','no'));\n\n\n\tif nextstepbf=='no' then return 'Not Ready'\n\telif and(diff >= 0,epubyn=='yes',bfyn=='yes') then return 'Up-to-date'\n\telif diff==-0 then return 'What does negative 0 mean'\n\telif and(diff<0,epubyn=='yes',bfyn=='yes',nextstepbf=='yes') then return 'Reupload'\n\telif and(bfyn=='no',nextstepbf=='yes') then return 'First Upload'\n\telif epubyn=='no' then return 'Needs EPUB'\n\telse return 'Unclear'\n\tfi;\n\n#\tif epub then\n#\t\tif ifempty(epub,'') && ifempty(bfupload,'') then\n#\t\t\tstrcat('EPUB Last Mod: ',epub,' (',days_between(today(),epub),' days ago) | Bookfusion Last Upload: ',bfupload,' (',days_between(today(),bfupload),' days ago) | Days Between EPUB Mod and BF Upload: ',days_between(bfupload,epub))\n#\t\telse\n#\t\t\tprint('Not all dates have values')\n#\t\tfi\n#\telse\n#\t\t''\n#\tfi;"
    ],

I'm not quite sure why any of these are triggering a problem now. Unless I only deleted the old #catalog_status column recently. I do recall deleting some columns I wasn't using, I just can't remember if that was one of them.

Can any of this stuff be deleted from the database to fix it? Or is there some other way to fix it?

Thanks.

[BetterRed]

AFAIK: if you delete a custom column the onus is on you to 'correct' any references to the custom column in save templates, virtual library searches, action chains etc.

BR

[kovidgoyal]

Use the calibredb command line tool to delete whatever custom columns you have that refer to the removed custom column. Or alternately just import all books from your old library into a new empty library without custom columns, by right clicking on add books and choosing add from folders and sub folders->one book per folder

This will preserve all basic metadata and any custom metadata for columns which you have created in the new empty library with the same name and type as in the old library.

[tamaracks]

Originally Posted by kovidgoyal

Use the calibredb command line tool to delete whatever custom columns you have that refer to the removed custom column. Or alternately just import all books from your old library into a new empty library without custom columns, by right clicking on add books and choosing add from folders and sub folders->one book per folder

This will preserve all basic metadata and any custom metadata for columns which you have created in the new empty library with the same name and type as in the old library.

Phew! calibredb seems to have done the trick. I deleted the saved search and a couple columns I thought were connected to the function using the missing column name, and then I was able to launch calibre with my library.

Thanks so very much for the help!