Edit Metadata in Bulk - not working (problem after update to 2.21.0)

[Rob557]

Yesterday I updated from 64-bit Calibre 2.13.0 to 2.21.0 (using Win7 on a Dell) and now find that Edit Metadata in Bulk does not work, even for very simple changes, although I can Edit Metadata on single ePubs.

When I try to use the edit bulk approach for a selection of books, everything seem to go normally but after I click on "ok" the data simply does not change. I tried even very simple match and replace bulk edits (using both character match and regular expressions) but nothing works.

Strangely enough, after I first installed the 2.21.0 update, I used the Edit Metadata in Bulk twice and it worked fine, but the third and subsequent times I tried it, it no longer worked.

This paragraph may be a clue to the problem or just an over-complicated red herring, so maybe ignore it for now if info in other paragraphs is sufficient: The only other anomaly that comes to mind is that after doing the update to 2.21.0, the first time that I tried to isolate a selection of books as a virtual library (specifying "current search" I think, as alway) I was prompted to name the virtual library. That had never happened before (I labelled it "temp"), but it's possible that I atypically clicked on "Create Virtual Library" (although I've never done that before) rather than "current search", since I was thinking of the new feature in 2.21.0 whereby edits within the tag browser can now be restricted to a selected virtual library. I then proceeded to isolate a different selection of books as a different virtual library (using "current search", with no prompt to name this virtual library) and then successfully performed the two bulk metadata edits referenced above. HOWEVER, then I decided to delete the previously-created "temp" virtual library since I would not need it later, and I think it was THEN that I was no longer able to do any more bulk metadata updates, even after re-boots, etc.

I also considered that the timing of the Calibre update was just a coincidence and maybe I was encountering some capacity limits because, over time I have gradually added more user-defined (30+) columns, including 3 more that I recently added. So as a test I deleted six of the columns that I am no longer using, and still find the metadata bulk edits do not work.

I have tried re-running the 2.21.0 update using the "repair" option, but it did not correct the problem.

I thought I would see others complaining about the problem, but that hasn't happened so maybe there is something unique in my circumstances.

Thanks in advance for any suggestions, or comments from others who might see if they now have the same problem.

[theducks]

V2.21 Bulk edit is working for me on W7 X64
Main (all) Lib, Selected from 2 books from the Search results

I suspect your VL has become corrupt (the needed naming, bit might be a clue.)
You might remove all VL's then run Library Maintenance

BTW I would avoid using using the typical reserved word names on General Principals :nods:
(learned this one the traditional hard way, quite some years back )

[Rob557]

Thanks for your comments theducks. I tried, but no solution as yet.

Originally Posted by theducks

You might remove all VL's then run Library Maintenance

I deleted one other virtual library that I had set up as part of my testing, and under Library Maintenance I ran "check library" (which said it did some "vacuuming" to improve performance, and then indicated no problems were identified), and then selected "check match" which again indicated no problems identified. I rebooted Calibre and tried a bulk metadata edit, but it is still not working.

Originally Posted by theducks

I would avoid using using the typical "reserved word" names

At the time that the problem occurred I was identifying ePubs that appeared to have OCR problems as indicated by the occurrence of multiple tildes "~" in the ePub, and was using the ~ symbol (is that a "reserved name"?) as part of the content added to the user-defined column. I've done that before though and didn't run into this problem. If it is a problem, "library maintenance" does not seem to provide a solution (even after I removed the other ~ references in that column by making rename-edits via the tag browser).

[theducks]

I don't remember if a tilde is a reserved character

The bigger part of issue is Calibre runs on 3 OS, each with their unique issues to avoid What was valid on one may not be on another.
Starting a name with a special character may have a different effect than use in the middle.
I really endorse using old style DOS, allowed filename characters. You never know when you will trip up a bug in one of the many building blocks used by Calibre with those on the edge

[BetterRed]

@Rob557 - did you try Library Maintenance->Restore database. It could be some library specific preferences (eg VL definitions) went awry, sometimes a database rebuild will fix. I would resolve any Check Library issues and backup my library before doing it.

On occasions I've had to edit the metadata_db_prefs_backup.json file to get rid of the 'malformed' data - I cant recall if it was VL or View Manager preferences. Take a copy of the file before you edit it - you can look at it when you're redoing the VL etc preferences/configuration.

AFAIK - tilde doesn't cause any problems within an EPUB, I use them frequently to indicate 'approximately' - eg "there are ~23M people in Australia".

And AFAIK the tilde has no significance in file names in the sense that <>:"/\|?* are disallowed in Windows. I use !#^~ at the start of folder names (not calibre library folder names) to force sorting into groups - bang, crunch, hat and squiggle indicate 'status' for my image, audio, video collections. I've used them for decades in DOS, Windows, Linux/Ext4 and Solaris/NFS - and probably OS2/HPFS, and various DEC OS's.

BR

[Rob557]

Thanks for your comments BetterRed.

Originally Posted by BetterRed

- did you try Library Maintenance->Restore database

I didn't do that ... and maybe it would have fixed the problem.

But as I was preparing to do that just now (after making a backup copy of the library), I tried a few bulk metadata edits as a benchmark for what to try after running the restore, and lo and behold the bulk edit process now seems to be working okay (without running the restore)!!

I don't get it. But for now the problem seems to have gone away. I wonder if the main metadata.db file has some sort of delayed self-correction as it continually updates itself ? I don't know at what point the problem went away because as I tried different things I kept testing to see if the bulk edit process was working. Then I ignored it and proceeded doing other work and work-arounds.

Originally Posted by BetterRed

On occasions I've had to edit the metadata_db_prefs_backup.json file to get rid of the 'malformed' data

That step is probably beyond my skill level. I tried a quick internet search and didn't find much. With windows explorer I can see the file in the Calibre directory (very small 46kb compared to the much larger multi-MB metadata.db file) and I can open and read the unformatted data stream with a basic text editor, but wouldn't know how to determine what was malformed data. It also seems risky to do any direct editing of the file and it seems more like an output data stream rather than a structured input data stream.

But I digress from the main topic in this thread and, inexplicably, the problem appears to have gone away. Hopefully it will not return. : / ...THANK YOU.

[BetterRed]

Hmmmm - that's three problems that have 'disappeared' for no apparent reason in recent days.

Rob557 - JSON files are 'unix' files (lines endings are LF only), sounds like you used an editor that wants 'dos' files (lines endings are CRLF), e.g. Notepad.

I use Notepad++, this is what I see - unix file on the left, dos file on the right.

BR

[Rob557]

Originally Posted by BetterRed

JSON files are 'unix' files ... I use Notepad++ (displaying) unix file on the left, dos file on the right.BR

I've just downloaded Notepad++ and can now see the structure, primarily the parameters for the columns in Calibre as well as recent search criteria etc. Thanks for the tip on Notepad++ BetterRed.