Most likely you are not. It is only there as a precaution in case you are still writing files to the drive, and chances are it has already finished any writing that calibre did to the drive. (Unless you try to pull the plug seconds after asking it to send 500 MB of books to the device.
)
But its usually better to be sure.
That being said, I usually *forget* to eject my flashdrives/kindles/usb memory storage of any kind before pulling it out, and I have never lost data. Although my flashdrive insists (on Windows only -- linux is perfectly happy) that there is an error and it wants to check for errors. I let it, each time it spends 25 minutes finding out nothing is wrong, then fails to clear the dirty bit for future reference. It then bugs me on subsequent mounts. Have I ever mentioned I hate Windows?