find and remove zero size .epub files?

hank · 06-19-2015, 01:22 AM

MacOS

At some point many months ago, I was converting .epub files to .pdb files -- which I read old Palm OS PDA -- and while I ended up with perfectly usable .pdb files, I also ended up with zero length .epub files.

I've just noticed the problem, because I was cleaning up the library (a total of over 3000 files, of which probably 300 or so have these zero length .epub files).

Yeah, the backups are the same. I don't use .epub except for converting to .pdb so hadn't noticed.

Can I find, and remove, the zero length .epub files in some way?

It's perfectly easy to see them with MacOS -- but I realize I shouldn't let the OS remove them. Or can I, and have Calibre forget about them or rebuild its record?

mbovenka · 06-19-2015, 05:24 AM

Quote:

Originally Posted by hank

Or can I, and have Calibre forget about them or rebuild its record?

You could do that and run Check Library afterwards and have it remove the missing formats, but there is an easier way: 'size:=0K and formats:epub' should find them.

hank · 06-19-2015, 11:36 AM

Quote:

Originally Posted by mbovenka

'size:=0K and formats:epub' should find them.

Pasting exactly that string as quoted into the search box finds nothing.
Using just 'formats:epub' does find all the books that include any epub.

Am I missing something about doing this in a sequence of steps?
Or, can anyone test this?
I'm one version behind, I'll update Calibre and try it again.

eschwartz · 06-19-2015, 11:53 AM

calibre still thinks they are the right size. Use the Quality Check plugin to disabuse it of that notion -- there is an option to fix incorrect format filesizes.

theducks · 06-19-2015, 12:04 PM

If these were small to begin with, the displayed value might just be rounding to 0

chaley · 06-19-2015, 12:38 PM

Also, if the book has more than one format, "size" is useless for this test. You would need to make a custom column built from other columns and populate it with the real file size for the epub using the formats_sizes() function.

hank · 06-19-2015, 06:45 PM

> 'size:=0K and formats:epub' should find them
Is that definitely the case, that search _should_ work? (under Mac OSX).
Because if that's broken somehow, I don't know what else I should mistrust about files.

Fix file size finds there are no file size problems.
Mac OS also sees these as zero size files.
(TextEdit sees them as blank pages too)

These did have measured file size info before I converted to get the .pdb files
I do suspect something during conversion trashed them.
Trying to open them with Mac OS 'get info' also shows zero size.

But I really don't need these .epub files now.
The .pdb files created from them do work fine and haven't lost any text.

And whatever the problem was that got the zero size files hasn't continued to happen. These are many months old.

I'd be happy to throw away the zero size .epub files.

Can I do that without opening all of several hundred book folders to find them and then closing the folder and using the "remove format" option one folder at a time.
What's the best way to do that?

How about using OSX to remove the zero size .epub files, then rebuilding the library, as suggested in the first response above?

hank · 06-19-2015, 06:49 PM

PS, I did get eight "OS artifacts" like these:
ERROR parsing book: /Users/hankrobots/Documents/Calibre-clean copy/Mark Twain/On the Decay of the Art of Lying (698)/On the Decay of the Art of Lying - Mark Twain.epub
Traceback (most recent call last):
File "calibre_plugins.quality_check.check_epub", line 733, in evaluate_book
File "site-packages/calibre/utils/zipfile.py", line 751, in __init__
File "site-packages/calibre/utils/zipfile.py", line 786, in _GetContents
File "site-packages/calibre/utils/zipfile.py", line 801, in _RealGetContents
BadZipfile: File is not a zip file

ERROR parsing book: /Users/hankrobots/Documents/Calibre-clean copy/Thomas Paine/Atheism refuted_ in a discourse to prove the existence of a God_ (1267)/Atheism refuted_ in a discourse to prove t - Thomas Paine.epub
Traceback (most recent call last):
File "calibre_plugins.quality_check.check_epub", line 733, in evaluate_book
File "site-packages/calibre/utils/zipfile.py", line 751, in __init__
File "site-packages/calibre/utils/zipfile.py", line 786, in _GetContents
File "site-packages/calibre/utils/zipfile.py", line 801, in _RealGetContents
BadZipfile: File is not a zip file

eschwartz · 06-19-2015, 07:00 PM

Sure. In Terminal.app:

Code:

cd /Users/hankrobots/Documents/Calibre-clean\ copy/
find . -name *.epub -type f -empty -delete

Then do Library maintenance ==> Check library

hank · 06-20-2015, 02:45 PM

OK, checked that Terminal string without the -delete, verified it's finding the right things, and ran it with -delete and did Check Library

> Check Library
I've told it to ignore file extension .DS_Store, and name .DS_Store
Check Library is reporting fourteen "unknown files in books" that are all .../.DS_Store
any problem with these?

hank · 06-20-2015, 03:00 PM

Hm, either that Terminal search missed some, or I've got new zero size .epub files (dated two days ago on the timestamp, apparently it missed some.

Odd.

hank · 06-21-2015, 02:52 PM

Now I have dozens again, all dated new since I cleaned them out and cleaned the library.
Is there some reason Calibre for OSX would be creating blank .epub files, maybe using empty files as placeholders or something?

eschwartz · 06-21-2015, 02:59 PM

No.

Are these books that used to exist as EPUBs, or are they books that didn't have EPUBs yesterday?

hank · 06-22-2015, 01:03 AM

These are books that I added as .epub files -- and two older copies of the library still have most of them with contents. I copy the whole library from time to time to back it up. I've just checked four different copies made over the past several weeks. For a given book, in some cases none of the .epub files have any content. Even the oldest of the four libraries has some zero size files as .epub. The newer ones have more of those files gone to zero size. And the newest has half a dozen .pdb files zero size.

Termites?

hank · 06-22-2015, 01:05 AM

I'm recovering some of the .epub books by moving the good copy forward to the latest library. That seems to be holding up so far.

I do find scattered blog posts over the last few years where people did get zero size files in Calibre for one reason or another (or, mostly, no known reason I can find). So it may be either misreading one of the instructions and doing something I should know better not to do -- or some rare bug not understood.

Dunno.

Oh, and there's no case I've found where a file that lacked an .epub acquired an empty .epub file (or any .epub file) spontaneously, if that's what you asked above.

06-19-2015, 01:22 AM	#1
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	find and remove zero size .epub files? MacOS At some point many months ago, I was converting .epub files to .pdb files -- which I read old Palm OS PDA -- and while I ended up with perfectly usable .pdb files, I also ended up with zero length .epub files. I've just noticed the problem, because I was cleaning up the library (a total of over 3000 files, of which probably 300 or so have these zero length .epub files). Yeah, the backups are the same. I don't use .epub except for converting to .pdb so hadn't noticed. Can I find, and remove, the zero length .epub files in some way? It's perfectly easy to see them with MacOS -- but I realize I shouldn't let the OS remove them. Or can I, and have Calibre forget about them or rebuild its record?

06-19-2015, 06:45 PM	#7
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	> 'size:=0K and formats:epub' should find them Is that definitely the case, that search _should_ work? (under Mac OSX). Because if that's broken somehow, I don't know what else I should mistrust about files. Fix file size finds there are no file size problems. Mac OS also sees these as zero size files. (TextEdit sees them as blank pages too) These did have measured file size info before I converted to get the .pdb files I do suspect something during conversion trashed them. Trying to open them with Mac OS 'get info' also shows zero size. But I really don't need these .epub files now. The .pdb files created from them do work fine and haven't lost any text. And whatever the problem was that got the zero size files hasn't continued to happen. These are many months old. I'd be happy to throw away the zero size .epub files. Can I do that without opening all of several hundred book folders to find them and then closing the folder and using the "remove format" option one folder at a time. What's the best way to do that? How about using OSX to remove the zero size .epub files, then rebuilding the library, as suggested in the first response above? Last edited by hank; 06-19-2015 at 07:00 PM.

06-19-2015, 06:49 PM	#8
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	PS, I did get eight "OS artifacts" like these: ERROR parsing book: /Users/hankrobots/Documents/Calibre-clean copy/Mark Twain/On the Decay of the Art of Lying (698)/On the Decay of the Art of Lying - Mark Twain.epub Traceback (most recent call last): File "calibre_plugins.quality_check.check_epub", line 733, in evaluate_book File "site-packages/calibre/utils/zipfile.py", line 751, in __init__ File "site-packages/calibre/utils/zipfile.py", line 786, in _GetContents File "site-packages/calibre/utils/zipfile.py", line 801, in _RealGetContents BadZipfile: File is not a zip file ERROR parsing book: /Users/hankrobots/Documents/Calibre-clean copy/Thomas Paine/Atheism refuted_ in a discourse to prove the existence of a God_ (1267)/Atheism refuted_ in a discourse to prove t - Thomas Paine.epub Traceback (most recent call last): File "calibre_plugins.quality_check.check_epub", line 733, in evaluate_book File "site-packages/calibre/utils/zipfile.py", line 751, in __init__ File "site-packages/calibre/utils/zipfile.py", line 786, in _GetContents File "site-packages/calibre/utils/zipfile.py", line 801, in _RealGetContents BadZipfile: File is not a zip file Last edited by hank; 06-20-2015 at 12:39 AM.

06-19-2015, 07:00 PM	#9
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Sure. In Terminal.app: Code: cd /Users/hankrobots/Documents/Calibre-clean\ copy/ find . -name *.epub -type f -empty -delete Then do Library maintenance ==> Check library

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Compare the font style and size from the source pdf with the destination epub files	kalees	ePub	0	01-22-2014 06:05 AM
Reducing the size of epub files?	Corpsegoddess	ePub	8	01-09-2013 08:14 PM
trying to read txt and epub files but can't even find them	bookmeal	Android Devices	2	11-04-2012 01:38 AM
How to find out which hack was applied? And how to remove it?	Starko	Kindle Developer's Corner	0	06-30-2011 01:43 PM
need option to remove iTunesMetadata.plist from epub files	jchrist	Calibre	5	09-16-2010 05:05 AM

06-19-2015, 11:53 AM	#4
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	calibre still thinks they are the right size. Use the Quality Check plugin to disabuse it of that notion -- there is an option to fix incorrect format filesizes.

06-19-2015, 12:04 PM	#5
theducks Well trained by Cats Posts: 29,863 Karma: 54837878 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	If these were small to begin with, the displayed value might just be rounding to 0

06-19-2015, 12:38 PM	#6
chaley Grand Sorcerer Posts: 11,757 Karma: 7029857 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Also, if the book has more than one format, "size" is useless for this test. You would need to make a custom column built from other columns and populate it with the real file size for the epub using the formats_sizes() function.

06-20-2015, 02:45 PM	#10
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	OK, checked that Terminal string without the -delete, verified it's finding the right things, and ran it with -delete and did Check Library > Check Library I've told it to ignore file extension .DS_Store, and name .DS_Store Check Library is reporting fourteen "unknown files in books" that are all .../.DS_Store any problem with these?

06-20-2015, 03:00 PM	#11
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	Hm, either that Terminal search missed some, or I've got new zero size .epub files (dated two days ago on the timestamp, apparently it missed some. Odd.

06-21-2015, 02:52 PM	#12
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	Now I have dozens again, all dated new since I cleaned them out and cleaned the library. Is there some reason Calibre for OSX would be creating blank .epub files, maybe using empty files as placeholders or something?

06-21-2015, 02:59 PM	#13
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	No. Are these books that used to exist as EPUBs, or are they books that didn't have EPUBs yesterday?

06-22-2015, 01:03 AM	#14
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	These are books that I added as .epub files -- and two older copies of the library still have most of them with contents. I copy the whole library from time to time to back it up. I've just checked four different copies made over the past several weeks. For a given book, in some cases none of the .epub files have any content. Even the oldest of the four libraries has some zero size files as .epub. The newer ones have more of those files gone to zero size. And the newest has half a dozen .pdb files zero size. Termites?

06-22-2015, 01:05 AM	#15
hank Dinosaur Posts: 107 Karma: 10 Join Date: Sep 2004 Device: Clie TH55, SJ30, VisorEdge, Palm3xe	I'm recovering some of the .epub books by moving the good copy forward to the latest library. That seems to be holding up so far. I do find scattered blog posts over the last few years where people did get zero size files in Calibre for one reason or another (or, mostly, no known reason I can find). So it may be either misreading one of the instructions and doing something I should know better not to do -- or some rare bug not understood. Dunno. Oh, and there's no case I've found where a file that lacked an .epub acquired an empty .epub file (or any .epub file) spontaneously, if that's what you asked above.

Advert

Advert