According to this thread from 2016,
it's only possible to search & replace a single book at a time. If it's changed since, I've not found the option for it.
I used Agent Ransack and DSR to select all the books in my Calibre library with the text string I want to replace. Now I'm not sure of the best way to go about doing this. The search box clears itself when opening a new file.
What other options are there? (Including third-party programs, if one exists.

)
EDIT: Since it was fanfic categories on the title page I was trying to change, I found that I can use FanFicFare to "update ePub always" will replace it with my properly regexed categories. If anybody still knows how to bulk search & replace that would be helpful.