What is your workflow to get books organized?

[Katsunami]

Even though I started another thread with a rant about how bad some ebooks are formatted, I must say that many *new* books (created since or after 2009 or so) DO seem to be getting better with regard to layout and coding.

Yesterday, I've bought a bunch of EPUB-books from three different stores; 17 books in total. (My goal in 2013 is to eventually re-read all paper books I have as an ebook, and then start selling the paper versions.) Not all stores had all the books I wanted, and sometimes the pricing differences were big: a series costing $6.95 per book in store A, cost $4.75 per book in store B; and for other series, it was the other way around. (And strangely enough, with two series, I had to split them between two stores, because none of them had the entire series. Why would you only carry book 1 and 4, but not 2 and 3?)

I've now looked into 7 of them, and their coding is clean and easy to understand, and their layout resembles the paper versions very much. These 7 books are all very well done; as the other 10 are all from the same publishers as these 7, I've got good hopes for them too.

Now, to get all this stuff organized.... what I do is the following.

1. Remove DRM by importing them into Calibre. Then I pluck the de-DRM-ed version from the library, and remove the books from Calibre again.
2. I save the deDRM-ed, but otherwhise unchanged versions to a separate folder, in case I need or want them someday in the future. I make copies to work with.
3. Used Sigil to remove spacing between the paragraphs of the body text. Calibre removes paragraph spacing everywhere AFAIK, and I don't want that. This is easily achieved by adding "margin-top: 0;" and "margin-bottom: 0;" to the right CSS-classes (and add "text-indent: 1em;" if necessary).
4. If the body text has a font-size set, then I remove this, so the font will not be larger or smaller than the size I've set in the e-reader.
5. Use EPUBMetadataEditor to set the tags as I want them, and to set the publication date to the original date of the paper version, instead of the date the ebook was published.

After editing the book as above, I'll save that version also, in my main library/folder. Then I import it back into Calibre, and everything immediately is set correctly.

I could do part 3-5 in Calibre itself, but I don't want to. I want to be able to, should it ever be necessary, to import my entire main library folder into Calibre and be done with it (except maybe ticking "Update sort order" for all books).

So in the end, I maintain two versions/libraries of my books:
1. The de-DRM-ed, but otherwise unchanged versions.
2. The de-DRM-ed versions I've adjusted to my liking.

There is also the library Calibre creates on its own, when importing the books from my main library folder; but I normally don't touch that.

Basically, I do the same with audio-CD's: I create my own FLAC version from an original CD (just as I create my own Ebook from an original EPUB). Then I normalize the volume to where I want it (ReplayGain), and check if the tags and cover image are correctly downloaded and set by the FLAC creator. Then I put this album into my music library, and import it into Foobar 2000. In the end, I archive the CD.

And of course I've got two backup copies of everything.

[Yolina]

Mine's far simpler

- import books in Calibre where they get the "Alf treatment"
- a quick tidy up: cover/medatada/insert jacket into books
- send books to Kindle(s)
- use the collections plug-in to automatically organise my stuff for me on the Kindle(s)

and that's it really (and yeah, 2 backups too ). If a book is really badly formatted, I delete it and ask for a refund, I don't really want to spend time fixing stuff (plus I'd need to learn how anyway...) If the publisher can't be bothered to make sure that the formatting is reasonable in the first place, then I can't be bothered to read it.

[Katsunami]

That's simpler indeed The big difference is that you do your tag editing in Calibre itself (i.e., in the Calibre database), and I do it outside, directly in the book itself. This means that I have to do the import twice: first to de-DRM the book (and then copy/remove it), and import again to get it into Calibre with the correct tags.

I could drop the Sigil part, but I don't like any whitespace in between each paragraph if I can avoid it. It costs me just a minute to add those two margin properties, at least in a WELL MADE BOOK, that is.

The newer ebooks I've been buying yesterday and the day before are much better than the older ones I bought some time back. Sometimes, a paper version gets a re-issue with a different cover, sometimes a different font, and that's about it; would they only change the cover on the ebook, or also update the layout and coding, if it was bad in the old version, I wonder? I'm not going to buy the same books in 2 or 3 years to check...

[Ripplinger]

Why do you bother removing the books from Calibre after you just loaded them in to remove the DRM?

I just open them in Sigil from the Calibre library location if I need to make changes. If I'm doing really extensive changes, I might make a copy of the book first in the same directory (but you have that covered since you save the copies elsewhere).

And I don't see the point with using EPUBMetadataEditor, everything you mentioned can be changed either in Sigil or Calibre.

And no, that doesn't mess up the Calibre library database at all, the book title stays the same.

I've recently bought a series where each chapter has a nice graphic across the top of the page. Except they used a slightly different version of the same graphic for each chapter, and each one positioned slightly different by a few pixels. Once I made each chapter use the same one image and deleted 37 now unused, the size of the epub was cut in half from over 700kb to less than 350kb. I just don't get their thinking when they do something like that, it's just senseless bloat for no reason. So I'm pretty finicky like you about how the epubs should look and edit quite a few of them myself.

[Katsunami]

Originally Posted by Ripplinger

Why do you bother removing the books from Calibre after you just loaded them in to remove the DRM?

I just open them in Sigil from the Calibre library location if I need to make changes. If I'm doing really extensive changes, I might make a copy of the book first in the same directory (but you have that covered since you save the copies elsewhere).

And I don't see the point with using EPUBMetadataEditor, everything you mentioned can be changed either in Sigil or Calibre.

And no, that doesn't mess up the Calibre library database at all, the book title stays the same.

Are you sure about this? If I change some of the metadata in the book, I don't think Calibre will pick that up. Is there some sort of "Read metadata", like Lightroom has?

With regard to usig EPUBMetaDataEditor: it's just something I've gotten used to. In Sigil, I need to add Metadata etries. in EPUBMetaDataEditor, I just run through it (and it has some handy buttons to copy stuff from one field to another), change what I need changed, and save.

I've recently bought a series where each chapter has a nice graphic across the top of the page. Except they used a slightly different version of the same graphic for each chapter, and each one positioned slightly different by a few pixels. Once I made each chapter use the same one image and deleted 37 now unused, the size of the epub was cut in half from over 700kb to less than 350kb. I just don't get their thinking when they do something like that, it's just senseless bloat for no reason. So I'm pretty finicky like you about how the epubs should look and edit quite a few of them myself.

With regard to size, I couldn't care less. I have FLAC files that are 30MB or bigger in size, so I don't mind an ebook that's 700 kB instead of 250 kB. I actually have some ebooks that are into the megabytes because they have maps, a big and different graphic for every chapter, graphical section seperations, and big graphical dropcap capitals for the first paragraph in a chapter.

[Ripplinger]

If all you're doing is settings tags as you want and the publication date to the original date of the paper version, instead of the date the ebook was published, that's easily done in Calibre through the Edit Metadata option. That's Calibre's metadata files.

As far as I know that information is not within the epub itself in any of the files. That extra info is always downloaded from other sources such as Amazon, Goodreads, etc.

And for myself, when the reader has 1.4GB space left after the OS, yes, I care if books are overly bloated for no reason. Don't get me wrong... I'd never delete maps, etc. as you say are sometimes included. I will compress them though when I see they're 500kb in size for each image when you compress them to under 100kb each or more. But really, the identical image for the top of a chapter... you don't need a different image file for each chapter. That's just wasting space stupidly. That I won't do.

[Katsunami]

Originally Posted by Ripplinger

If all you're doing is settings tags as you want and the publication date to the original date of the paper version, instead of the date the ebook was published, that's easily done in Calibre through the Edit Metadata option. That's Calibre's metadata files.

As far as I know that information is not within the epub itself in any of the files. That extra info is always downloaded from other sources such as Amazon, Goodreads, etc.

I know Calibre can do it using it's database, but I want it to be in the EPUB itself. That way, I never have to set anything in Calibre, or, should it come to that, any other ebook organizer in the future.

And for myself, when the reader has 1.4GB space left after the OS, yes, I care if books are overly bloated for no reason. Don't get me wrong... I'd never delete maps, etc. as you say are sometimes included. I will compress them though when I see they're 500kb in size for each image when you compress them to under 100kb each or more. But really, the identical image for the top of a chapter... you don't need a different image file for each chapter. That's just wasting space stupidly. That I won't do.

In that regard, we differ Things like graphical chapter headings, dropcaps, maps, and that sort of stuff make an eboook an alternative to a normal book. I detest it when the publisher tries to do funny things with the normal text (such as locking font choices, setting small fonts for body text and such), but I like the extra's such as the dropcaps and so on.

I have no qualms about maps and covers that are 500 kB in size. Actually, if I deem a map or cover too small or not good enough in quality, I'll find a bigger version on Google and replace it. Because I never put more than a few books onto the reader, so space is no problem. Therefore I don't compress graphics.

[latepaul]

I recently automated some of my work-flow as I was getting annoyed with repetitive manual bits.

However I suspect my needs are simpler

So the process is:

1. Download the purchased file. Generally I buy from either Amazon or Kobo. If Kobo then I'll download the Adobe ePub (though I will let my Kobo devices download the kepub files).
2. Depending on the source I strip the DRM. I don't use the Calibre plugins because I don't want to have to import, delete, re-import. Without going into detail (which would be bad) I do the Amazon files on a Windows PC but the epubs on my Linux desktop. Each has an output directory with the DRM-free files. If on Windows I copy it to the Linux PC for the next stage.
3. This is the bit I automated. Well semi-automated. I run a shell script I wrote against each file. This imports into Calibre and lets me set the custom metadata fields I use. It also lets me set series and series index. It checks against the calibre database and gives me a little menu of existing series, though I can add new ones. Oh and it also lets me correct Author name as this is often in the form "Surname, Forename" but Calibre has Author_Sort for that so I change it if needed. As a final step it copies the file to an archive directory - this is my repository of orginal files. If I later edit them or re-import then into Calibre I'll do so from here.
4. Run them through a few plugins - Count Pages, Goodreads sync and Reading List. The later means they'll automatically get added to my devices during next sync.
5. Run my backup script which will back up my original files, the repository and the Calibre directory (which is also on Dropbox).

And that's it. I don't as a matter of course edit my ebooks unless there's somethine particularly egregrious there. I also don't update the metadata of the files except by default because I'll usually end up doing either mobi/AZW3->epub conversion or vice-versa. I have bought a few anthologies and collections and I have split those up into separate books with Sigil.

So you can see what I mean about simpler needs.

[DiapDealer]

Originally Posted by Katsunami

I know Calibre can do it using it's database, but I want it to be in the EPUB itself. That way, I never have to set anything in Calibre, or, should it come to that, any other ebook organizer in the future.

Meme's Modify ePub plugin.

Also the new "Polish Books" feature introduced in v0.9.19 (that's "shoe" polish ... not "from Poland" Polish).

Both will allow you to update the internal metadata of the epub file(s) with the data you've so lovingly spent hours and hours--hours that you could have spent actually reading those books--perfecting in calibre.

My procedure is : buy it, strip it, squirrel a few backups away, and read it. All done!

[pidgeon92]

All I ever to is download the books and import them into calibre (liberating them in that process). I also add the title to Delicious Library, as I use that to keep track of all of my reading, including library books and audiobooks.

If there isn't a book cover, I add it in calibre, but I don't bother to reconvert the book, as I don't really care if the book cover appears on my ereader. Nor am I concerned about the formatting. It has to be really awful for me to bother trying to alter it. I've altered the formatting on perhaps 3 ebooks out of the hundreds I've read since 2008.

Unless I'm reading a series, I generally pick one book from my catalog at random to read next. I'm far too lazy to tag/categorize all of my ebooks.

[Geralt]

Workflow? Organizing? What's that?
Seriously I download the book, load it in to Calibre and send it to kindle.
Or if I buy on amazon I turn on the wi-fi and download directly to kindle.
The time you spend organizing, tagging, editing I spend reading

[Katsunami]

Originally Posted by alucarda

Workflow? Organizing? What's that?

After someone asks you to play a certain CD or show you a certain picture, and you know you have it on your computer but are unable to find it, you'll start to know what "workflow" and "organization" mean.

I have hundreds of CD's, and thousands of pictures, and want to be able to find them all within seconds if I need or want one. Same with books, as I'm starting to accumulate a lot in a short time, because I'm reading much more than last year. (I didn't really use the Kindle Touch that much in the year I had it.)

I hate a cluttered computer. I've also got many DVD's neatly sorted by title, in a nice book case. I don't throw a few hundred DVD's (or paper books, or CD's) at random in a closet or on the floor; so why would I do that with files on my computer, especially when I paid good money for them, and certainly don't want to (accidentally) lose them?

The time you spend organizing, tagging, editing I spend reading

This workflow takes me about 2, maybe 3 minutes per book, so I can add 20-30 perfectly tagged books in calibre per hour. I never bought 20 or 30 books in a day. Buying 17 books at once yesterday was a (big) exception.

Often, I rip, tag and organize multiple CD's and multiple books at the same time. Doing a few each day, which takes no more than 15-30 minutes, nets you a very organized collection after some time

[Geralt]

I dont have any CD's and I dont get asked to show pictures to people. Well except online, and then they ask for boob pictures, so that's a no go.
My PC files are well organized and I know where my pictures/music/documents are. I dont do any of that stuff you do with ebooks. The procedure you described in your opening post is closer to nitpicking then organizing imo.
Btw, I was joking, no need for you to assume that just because I dont fiddle with ebooks I have a messy PC. One doesnt imply the other you know.
But what you described of doing is as Japanese would say 面倒くさい。

[elcreative]

I'm with alucarda... and I use the computer for finding/organising - that's one of the things they save people wasting their time on - and as for books, they're self-organising - author, title, series and primary grouping, anything else needed is fine but definitely personal and I'd rather read than be bothered... and I rarely have problems with several terabytes of data... Time to read, photograph, improve my artwork etc. is far more useful than time wasted on something of no value to me... for those who need it, fine but for the rest of us then "good enough" is "good enough" and we can get on with what we value and find important... each to their own life...

[travger]

Originally Posted by Ripplinger

As far as I know that information is not within the epub itself in any of the files. That extra info is always downloaded from other sources such as Amazon, Goodreads, etc.

Wrong. I drag a book into calibre and at once get all kinds of fields filled: Author, Title, Tags, Publisher, Publishing Date... All this info comes directly from file contents (as set in my prefs). If I feel like it, I download metadata from Calibre button and see what it offers.
Drawbacks are that I get too many tags (General and Fiction are constantly reappearing on my tags list), only those fields I have common with other people get filled (Series, but not CoverArtist).
So yes, first checking metadata and then adding to Calibre makes sense to me.

I'm not sure how much of it could be achieved from Calibre and Sigil. There's a 'i' button on Edit panel that reads metadata from the file, so maybe one step OP could save is re-adding. Just leave it in Calibre, do your thing on the other copy and then drag into book Book Details panel. Open Edit and hit 'i'.