Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-17-2015, 11:47 PM   #1
Colerson
Nameless Being
 
Determining original format of book

As a former Nook user, I have many books with both an AZW3 (or MOBI) format that I bought from Amazon, as well as the EPUB I converted to in order to put the book on the Nook.

Now, with a Kindle, I have EPUBs that I bought from B&N which I'm converting to AZW3, so as with the Nook books, I have both AZW3 and EPUB formats.

If I'm doing a reconversion (to fix paragraph spacing or something), I always want to use the original format. However, with the scenario outlined above, the only way I can know the original source is if I remember which store I bought a book from (or look it up in my purchase history).

Is there an easy way to determine which format is the “original”, by which I mean the first imported into Calibre? Basically, given a book with an AZW3 and EPUB, I'd like to know which came first.

If there's no easy way in the UI, is it a safe assumption that, in the “data” table in metadata.db, the entry with the lowest primary key value is the original?
  Reply With Quote
Old 01-18-2015, 07:32 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Just look at the last modified timestamp of the actual files, which you can get by hovering over the format in the upper right corner of the edit metadata window.
kovidgoyal is offline   Reply With Quote
Advert
Old 01-21-2015, 07:15 AM   #3
Agama
Guru
Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.
 
Agama's Avatar
 
Posts: 776
Karma: 2751519
Join Date: Jul 2010
Location: UK
Device: PW2, Nexus7
Quote:
Originally Posted by Colerson View Post
Is there an easy way to determine which format is the “original”, by which I mean the first imported into Calibre? Basically, given a book with an AZW3 and EPUB, I'd like to know which came first.
You may then want to use Kovid's method as a basis for populating a custom column - provided you don't have too many books. My situation was similar, (EPUB, MOBI and AZW3), so I now use a custom column to show the original format. This makes the data easily accessible at a glance and allows you to filter your library view based on original format.
Agama is offline   Reply With Quote
Old 01-23-2015, 01:21 AM   #4
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
If a book was bought from Amazon, it will have an exth 113 field, which contains the ASIN (product identifier). When calibre imports the AZW* file, it will add this ASIN as the identifier "mobi-asin".

If a book has the mobi-asin identifier, it is a pretty good bet that the book was originally from Amazon.

Of course, I also add a tag for any book that isn't from Amazon, to indicate where it came from. (Tags are useless, so I use them to keep track of other information, like original source, if the book is an omnibus/anthology, and whether or not any metadata or editing work needs to be done.)

Last edited by eschwartz; 01-23-2015 at 01:23 AM.
eschwartz is offline   Reply With Quote
Old 01-23-2015, 11:34 AM   #5
Colerson
Nameless Being
 
Ok, thanks a lot for the suggestions.

Because I'm rather lazy, I hacked Calibre so that in the book details view, the original format has a small note next to it (e.g. it looks like “Formats: AZW3, EPUB (original)” now). More work up front that will hopefully lead to less work over time.

There was one interesting issue: different versions of Calibre seem to have done different things in the “data” table.

Later entries (newer versions of Calibre), when creating an ORIGINAL_<format> entry, appear to rename the original entry and add a new one, so EPUB becomes ORIGINAL_EPUB and a new entry for EPUB is created, with a higher id.

Earlier entries (older versions of Calibre) appear to just create a new entry for the ORIGINAL_<format> entry, leaving the original intact. Thus ORIGINAL_EPUB for a book has a higher id than EPUB, opposite of the later entries.

In the end it's immaterial which is listed since they mean the same thing, but it did confuse me for a minute when I saw the apparent inconsistencies across books.
  Reply With Quote
Advert
Old 01-23-2015, 12:02 PM   #6
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Those are (usually) totally different.
Your 'Original' is your what was first imported
The 'Original_*' is what (of the same format) things started with.

eg
You start (original) with Mobi
You convert to EPUB (still no 'Original_*')
You Polish the Epub. Presto! Original_Epub'

There are other same-format actions that will create the Original_*
There are Tweaks (save_original_format) to control which happen

BTW I use the words 'Master Format' to describe the true original. I also archive this outside of Calibre, just in case
theducks is offline   Reply With Quote
Old 01-23-2015, 12:59 PM   #7
Colerson
Nameless Being
 
Quote:
Those are (usually) totally different.
Your 'Original' is your what was first imported
The 'Original_*' is what (of the same format) things started with.

eg
You start (original) with Mobi
You convert to EPUB (still no 'Original_*')
You Polish the Epub. Presto! Original_Epub'
Yes, when I say “original” format I mean the initial format which was imported. In fact, I think it makes more sense to call it “initial format” to avoid overlap and confusion with Calibre's idea of what “original” means; thanks for noting the terminology problem.

In my case, I almost always want to convert from the initial format, regardless of what has been done in the meantime, and that's what my patch allows me to do. In fact, I've modified it so that the conversion dialog always chooses the initial format by default, regardless of preferences or saved conversion settings.

So in the above example, MOBI is the “initial format”, tagged as such in the GUI, and used as the source for conversions, all without manual intervention. I don't claim this is useful for the majority, but it is the workflow that I prefer.
  Reply With Quote
Old 07-20-2015, 02:48 PM   #8
Cyberman tM
Evangelist
Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.Cyberman tM ought to be getting tired of karma fortunes by now.
 
Cyberman tM's Avatar
 
Posts: 439
Karma: 2248782
Join Date: Aug 2009
Location: Austria
Device: Inkbook Prime; Icarus Illumina;ImcoV6l;EB600;Kobo
Is this thread still up to date? I searched for "original format" and found this thread.

Are there plans to add in a "initial format" marker? I'd really like to have that as well. Just bought another book on Amazon that I immediately converted to ePub. Timestamp is different, of course, but just for a few minutes...

IMO it would be great if the initial import format could be simply highlighted by bolding it in the right-hand information display.

(I'll survive without it, of course. Calibre is doing a great job already.)
Cyberman tM is offline   Reply With Quote
Old 07-20-2015, 03:24 PM   #9
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by Cyberman tM View Post
Is this thread still up to date? I searched for "original format" and found this thread.

Are there plans to add in a "initial format" marker? I'd really like to have that as well. Just bought another book on Amazon that I immediately converted to ePub. Timestamp is different, of course, but just for a few minutes...

IMO it would be great if the initial import format could be simply highlighted by bolding it in the right-hand information display.

(I'll survive without it, of course. Calibre is doing a great job already.)
AFAIK there was never a plan to do this for the small number of us that keep track. I still set it (post 6) as part of my intake workflow.
theducks is offline   Reply With Quote
Old 07-24-2015, 03:19 PM   #10
Colerson
Nameless Being
 
Quote:
Originally Posted by Cyberman tM View Post
Is this thread still up to date? I searched for "original format" and found this thread.

Are there plans to add in a "initial format" marker? I'd really like to have that as well. Just bought another book on Amazon that I immediately converted to ePub. Timestamp is different, of course, but just for a few minutes...

IMO it would be great if the initial import format could be simply highlighted by bolding it in the right-hand information display.

(I'll survive without it, of course. Calibre is doing a great job already.)
I've made a patch for Calibre which add a new “initial format” column and automatically populates it whenever a new book is added. It also includes a patch to automatically use the initial format as the preferred input source when converting.

I've attached the patch here: the first part does the actual creation of the column and adds the books to it, the second does the conversion selection. You can remove the second part and still get the initial format column.

This is a source patch, so it only works if you're building Calibre yourself, or possibly you can patch installed .py files (I don't know how Calibre is packaged for various operating systems, as I always build from source). It's quite possibly more trouble than it's worth for most people.
Attached Files
File Type: txt 01-initial-format.txt (1.7 KB, 116 views)
  Reply With Quote
Old 07-24-2015, 05:49 PM   #11
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
You can patch the library.zip, I think, on Windows. On linux everything is in /opt/calibre/lib/python2..7/site-packages/calibre, but unfortunately for patching, there is only *.pyo files there.
You could download the right version from github, patch that, and optimize it (or not), then put that in.

You can also do: http://manual.calibre-ebook.com/develop.html which is honestly a lot easier.


P.S. If you build from git sources, you can do a:
Code:
python2 setup.py bootstrap # do build, resources, and translations -- plus dependencies
sudo python2 setup.py develop
The re-run the bootstrap whenever changes touch the C++ extensions.
Yes, I do that.
I asked for that bootstrap command, it is fairly new. Up until now, building resources/translations has been a bit mysterious unless you use the versioned tarballs.

Last edited by eschwartz; 07-24-2015 at 06:07 PM.
eschwartz is offline   Reply With Quote
Old 07-24-2015, 11:07 PM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Just a word of warning, dont use this patch, it modifies the calibre database schema, which is not a safe thing to do.

@Colerson: While I am assuming this patch is really "for your own use", so you dont want to spend too much time on it -- I'd suggest using add_custom_book_data()/get_custom_book_data() instead of adding a column to the db. And you have to deal with the case when the book being added has multiple formats, which can happen during a recursive import (right clicking the add books button). And you'd need to provide some UI to allow the user to change the "initial format", since you are basing conversions off it.
kovidgoyal is offline   Reply With Quote
Old 07-25-2015, 04:37 PM   #13
Colerson
Nameless Being
 
Quote:
Originally Posted by kovidgoyal View Post
Just a word of warning, dont use this patch, it modifies the calibre database schema, which is not a safe thing to do.

@Colerson: While I am assuming this patch is really "for your own use", so you dont want to spend too much time on it -- I'd suggest using add_custom_book_data()/get_custom_book_data() instead of adding a column to the db. And you have to deal with the case when the book being added has multiple formats, which can happen during a recursive import (right clicking the add books button). And you'd need to provide some UI to allow the user to change the "initial format", since you are basing conversions off it.
It was very quick and dirty, mainly done when I switched from Nook to Kindle, to ensure I was always using the original source in cases of e.g. epub and mobi existing, and not knowing which was original. By using a custom column, the UI is already there, and I wasn't concerned with recursive imports.. but since there is some desire from others for this functionality, I may look into doing it in a more proper fashion.
  Reply With Quote
Old 07-26-2015, 01:54 PM   #14
dwig
Wizard
dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.
 
dwig's Avatar
 
Posts: 1,613
Karma: 6718479
Join Date: Dec 2004
Location: Paradise (Key West, FL)
Device: Current:Surface Go & Kindle 3 - Retired: DellV8p, Clie UX50, ...
Quote:
Originally Posted by kovidgoyal View Post
Just a word of warning, dont use this patch, it modifies the calibre database schema, which is not a safe thing to do.

@Colerson: While I am assuming this patch is really "for your own use", so you dont want to spend too much time on it -- I'd suggest using add_custom_book_data()/get_custom_book_data() instead of adding a column to the db. And you have to deal with the case when the book being added has multiple formats, which can happen during a recursive import (right clicking the add books button). And you'd need to provide some UI to allow the user to change the "initial format", since you are basing conversions off it.
And to be really solid, it needs to deal with merged books where more than one file is an "original", one from each of the merged records. And, of course, it needs a way to manually fix the record when the code's decision as to which to call the "original" is not the user's choice. The fix would need to include the option to choose no original on an individual record basis.
dwig is offline   Reply With Quote
Old 07-26-2015, 10:41 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
For that matter it has to deal with ORIGINAL_FMT also when doing a FMT to FMT conversion.
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to tell the original book format? bald Calibre 9 10-13-2014 07:00 AM
Best original format for accurate conversion ren41 Conversion 8 10-18-2011 10:02 AM
Tracking original / source format? threejosh Library Management 10 01-28-2011 08:47 PM
PRS-350 Had to format, need original folder names ardeegee Sony Reader 9 01-17-2011 11:30 PM
Best Original Format for Kindle fading Amazon Kindle 6 09-08-2009 09:25 PM


All times are GMT -4. The time now is 07:44 AM.


MobileRead.com is a privately owned, operated and funded community.