When the book browser is displayed and you minimize the main window the book browser remains maximized with no way to minimize it.
I managed to duplicate this. I also found the solution. With the book browser window showing on screen and the rest of Sigil minimized, double click the book browser titlebar. It then disappears and ends up back to being docked.