I just made a test with a book I downloaded from archive.org: 291 pages pdf = 5 MB. After I OCR'd it with Adobe Acrobat DC Pro 2019, I got a filesize of 200 MB which however works smoothly with Koreader. After that I made another OCR with Abby Finereader 12 which compressed it to 10 MB but again caused heavy delay in Koreader. So that was the reason for my freezes and crashes. I guess I'll stick to bigger files now. Markismus' method would be an alternative.
Another remark about uneven margins, the page break (setting: page crop "auto") looks awkward
When a book is really important to me, I crop all the margins with ScanTailor. But I wonder if the Koreader could automatically adjust each page break