I think the reason is that a file which is in the Library is in a protected storage area so that the user cannot delete/move it without the Library's knowledge. This allows the Library to keep the annotations, which I believe are stored separately from the PDF/ebook, properly associated with the ebook/PDF.