OP here, I resorted to using export Text in WinDJView.
This gets you a text dump with no formatting whatsoever. For my Libre it works well enough, but in general, this procedure is suboptimal.
Most DJVU files do seem to have a text layer (unless there is some on the fly OCR happening when you select an area on the page, which seems unlikely).
Thus, there must be a way(at least theoretically until someone writes a converter) to preserve the formatting in the text layer.