Embed Metada in PDF

[p0lrb3ar]

I have an issue where I select 100 PDF files to embed metadata. If one of the files is broken, then the embed STOPS processing everything else, but could take 10 minutes to actually seem like it finishes. Only to display an error.

I know it stops, because I am testing with some known bad PDFs to verify the behavior. I would expect an error list of broken files while the rest succeed, but the behavior is that Calibre Embed appear to continue but actually not proceed past the first broken file. I have to fix the file then reselect the PDFs and start the embed process again to continue.

The whole process breaks after the error, but seems to simulate continuing, doesn't collect any more logs for other broken PDFs and doesn't LOG ANYTHING in the DEBUG log. This program really needs better logging.


calibre, version 8.12.0
WARNING: Failed for some files: Failed to embed metadata into some book files. Click "Show details" for details.

The PDF format of Applied Mathematics in Engineering:

Traceback (most recent call last):
File "calibre\utils\podofo\__init__.py", line 36, in set_metadata
File "calibre\utils\ipc\simple_worker.py", line 251, in fork_job
File "calibre\utils\ipc\simple_worker.py", line 176, in run_job
File "calibre\utils\ipc\simple_worker.py", line 119, in communicate
calibre.utils.ipc.simple_worker.WorkerError: Worker failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\customize\ui.py", line 529, in set_file_type_metadata
File "calibre\customize\builtins.py", line 531, in set_metadata
File "calibre\ebooks\metadata\pdf.py", line 193, in set_metadata
File "calibre\utils\podofo\__init__.py", line 40, in set_metadata
Exception: Failed to set PDF metadata in (Applied Mathematics in Engineering): Traceback (most recent call last):
File "calibre\utils\ipc\simple_worker.py", line 299, in main
File "calibre\utils\podofo\__init__.py", line 97, in set_metadata_
podofo.Error: PdfErrorCode::NoEOFToken, No EOF Marker was found in the PDF file.
Callstack:t#0 Error Source: main\PdfParser.cpp(97), Information: Unable to load objects from file
t#1 Error Source: main\PdfParser.cpp(143), Information: Unable to load xref entries
t#2 Error Source: main\PdfParser.cpp(356)
File: main\PdfParser.cpp Line: 97 Unable to load objects from file
File: main\PdfParser.cpp Line: 143 Unable to load xref entries
File: main\PdfParser.cpp Line: 356

[p0lrb3ar]

This one took about ten minutes to fail. I'm thinking there should be a cursory check if the PDF is locked or protected in any way, then a shorter timeout period possibly?

I can post more examples if needed:

calibre, version 8.12.0
WARNING: Failed for some files: Failed to embed metadata into some book files. Click "Show details" for details.

The PDF format of Oxford Handbook of Respiratory Medicine 4E:

Traceback (most recent call last):
File "calibre\utils\podofo\__init__.py", line 36, in set_metadata
File "calibre\utils\ipc\simple_worker.py", line 251, in fork_job
File "calibre\utils\ipc\simple_worker.py", line 176, in run_job
File "calibre\utils\ipc\simple_worker.py", line 104, in communicate
calibre.utils.ipc.simple_worker.WorkerError: Worker appears to have hung

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\customize\ui.py", line 529, in set_file_type_metadata
File "calibre\customize\builtins.py", line 531, in set_metadata
File "calibre\ebooks\metadata\pdf.py", line 193, in set_metadata
File "calibre\utils\podofo\__init__.py", line 40, in set_metadata
Exception: Failed to set PDF metadata in (Oxford Handbook of Respiratory Medicine 4E):

[kovidgoyal]

Those are not locked or protected, they are corrupted. And embed metadata does show a list of failed books after the process completes. If you arent getting that, something else is going on. Try rebooting in safe mode and running the embed.

[p0lrb3ar]

Thanks for the response.

I am able to open and read the PDF files. One of them was not editable in FOXIT, so I set it to allow, which allowed the Calibre embed to work. The other was fine and readable, and passed a QPDF check. I was able to edit the metadata with Stirling PDF. So, not knowing what was happening I used ghostscript to rebuild it, then it worked with Calibre.

Both of these were part of a group of 10 PDFs that I selected, after the failure the log only showed the first failure, not both. Next time I will open each PDF in the list and check if the embed feature continued to write the remaining files. One thing I noticed is that the embed is only appending authors not overwriting them. So, a PDF with lastname, firstname would end up being "firstname lastname; lastname, firstname" after the embed.