Calibre 0.7.x supports sending books to iTunes/iDevices in OSX and Windows. This thread contains in-depth information about using iTunes with calibre.
If your regular Reader is a device other than an iDevice (Sony, Kindle, Nook, etc) and you routinely connect an iPhone to your computer to charge or sync it with iTunes, you will want to disable your iDevice in calibre. SeeIf you have problemsbelow for details.
If you have recently updated your version of iTunes (e.g., from 9.x to 10.x), it is important to let iTunes update its database before connecting to it with calibre.
Quick start: 'Connect to iTunes' mode
Launch iTunes with no connected iDevices.
In iTunes Preferences|Advanced, uncheck Copy files to iTunes Media folder when adding to library. This tells iTunes to use calibre's copies of your books.
Launch calibre, set Preferences:
Preferences|Behavior|Preferred output format to EPUB
Preferences|Conversion|common options|page setup|Output profile to iPad
Preferences|Plugins|Device Interface plugins|Apple device interface - click Enable/Disable plugin so that driver is disabled (grayed out)
Restart calibre.
Select Send to device↓Connect to iTunes to connect iTunes as a Reader device.
To send epubs or PDFs from calibre to iTunes:
Select the book(s) to send from your calibre Library window, then click Send to device. Your books are sent to 'Books' in your iTunes library.
To use Fetch news to download periodicals from the Internet:
Preferences|Interface|Automatically send downloaded news to ebook reader: Recommendation: checked
Preferences|Interface|Delete news from library when it is automatically sent to reader: Recommendation: unchecked
Click Fetch news, select the recipe you wish to download, click Download now. Repeat for other recipes, then click Save to exit the Schedule news download dialog.
When you're ready to sync your iDevice with iTunes:
Connect your iDevice to your computer
Sync your iDevice with iTunes as you normally would.
If you manually manage songs and videos in iTunes, you can simply drag books from the Library|Books list to your iDevice.
Options
Configuration options are accessed in Preferences|Plugins|Device Interface plugins|Apple device interface, then clicking Customize plugin.
Select available formats and their order for this device: If you have both an epub and pdf file, the order of these formats in this list determines which will be sent to iTunes when clicking Send to device. If you want to send a particular format, use Send to device↓Send specific format.
Use Series as Category in iTunes/iBooks: If checked, the Series name from the book's metadata will be used as the Genre in iTunes. If no Series name exists, the first tag will be used as the Genre. If unchecked, the first tag will be used as the Genre.
Cache covers from iTunes/iBooks: If checked, calibre will read cover images from iTunes and display them in the Book Details section of the Reader pane. Calibre caches cover images to improve performance. If you have a very large library and simply want to load as fast as possible, uncheck this option.
Metadata Mapping and Plugboard Transformations
Spoiler:
Calibre sends eBooks to iTunes/iBooks with its best effort to apply the book's metadata as it exists in the calibre database. The relationship between calibre metadata and iTunes metadata is not exact. Metadata items listed as 'tweakable' may be transformed via a plugboard. Plugboard transformations are discussed below.
Calibre metadata may be viewed in calibre's Edit Meta information dialog, invoked by clicking the 'i' icon or pressing the 'E' key.
iTunes metadata may be viewed in iTunes' Get Info dialog. iBooks metadata is implicit in the Library list view, where books may be sorted by Titles, Authors, or Categories.
Title
calibre: Title
iTunes: Name, Album
iBooks: Title
tweakable: Yes
See Title transformations below for an example of prepending Series information to Title when displayed in iTunes/iBooks.
Sort Title
calibre: title_sort (by default, not visible in UI**)
iTunes: Sort Name, Sort Album
iBooks: Sort Title
tweakable: Yes
The algorithm generating Sort Title looks for opening articles 'A', 'An' or 'The', moving them to the end of the title. E.g., 'The Man Who Loved Books Too Much' becomes 'Man Who Loved Books Too Much, The'
Author
calibre: Author(s)
iTunes: Artist
iBooks: Author
tweakable: Yes
See Display Authors as LN, FN in iTunes/iBooks below for an example of using Author Sort in place of Author.
Author Sort
calibre: Author Sort
iTunes: Sort Artist
iBooks: Sort Author
tweakable: Yes
Publisher
calibre: Publisher
iTunes: n/a
iBooks: n/a
tweakable: No
Plugboard transforms to 'publisher' are ignored.
Tags
calibre: Tags
iTunes: Genre
iBooks: Category
tweakable: Yes
The first tag beginning with an alphabetic character is used as the iTunes Genre / iBooks Category. If no tags, the first <dc:subject> from the ePub's OPF file is used. See Using a custom column as Genre/Category below for an example of directly specifying Genre/Category for iTunes/iBooks.
Series Information
calibre: Series/series_index
iTunes: Sort Name, Genre, Episode ID, Episode Number
iBooks: Sort Title, Category
tweakable: Yes
If a book has series information AND Preferences|Advanced|Plugins|Device Interface Plugins|Apple device interface|Customize plugin|Use Series as Category in iTunes/iBooks is checked, the Sort Title is generated from the Series/series_index, and the Category/Genre is set to the Series name. Series and series_index are passed as Episode ID and Episode Number. N.B.: Tag transformations override using Series as Category/Genre.
Comments
calibre: Comments
iTunes: Comments
iBooks: n/a
tweakable: No
Comments are clipped at 255 characters by the iTunes automation interface. Longer comments may be added manually. Comments are not visible in iBooks.
Language
calibre: language (not visible in UI)
iTunes: added to ePub's OPF file
iBooks: added to ePub's OPF file
tweakable: Yes
Plugboard transforms to 'language' are added to the ePub's OPF file as <dc:language>.
UUID
calibre: uuid (by default, not visible in UI**)
iTunes: Composer
iBooks: Composer
tweakable: No
The uuid is used to correlate books between the calibre library and the iTunes database. Its use allows the metadata attributes to be different while still allowing the book to be recognized as the same.
Description
calibre: n/a
iTunes: Description
iBooks: Description
tweakable: No
A comment is added to iTunes/iBooks of the form 'added by calibre 2010-10-05 11:41:31'.
Transforming metadata using calibre's Metadata plugboard feature
Added in 0.7.21, plugboards offer users the opportunity to transform metadata as an eBook is saved to disk or sent to a device. This discussion focuses exclusively on plugboarding with iTunes/iBooks. Plugboard support for iTunes/iBooks was added in 0.7.23.
Title transformations
A common use of plugboards is transforming a book's title when it is part of a series. For example, consider 'The Girl with the Dragon Tattoo' by Stieg Larsson, book 1 of the Millenium series. By default, iBooks would list it as simply 'The Girl with the Dragon Tattoo'.
To have the book listed in iTunes/iBooks as 'Millenium 1: The Girl with the Dragon Tattoo':
Open the Metadata plugboards dialog - Preferences|Import/Export|Metadata plugboards.
Select any format, epub or pdf from the Format dropdown list, then APPLE from the Device dropdown list.
In the Source template field, add this template:
Code:
{series}{series_index:| |: }{title}
This tells the templating engine "If there is a series_index for the book, prepend the series name, a space, the series_index, then a colon and a space before the book's title."
Select title from the Destination field dropdown list.
Click the Save plugboard button, then Apply (upper left corner of the dialog) to save and close.
All books matching the specified formats sent to iTunes/iBooks will have this plugboard transform applied.
Using a custom column as Genre/Category
Creating a custom 'Genre' column is useful when you want exact control of the Genre/Category applied in iTunes/iBooks:
In Preferences|Interface|Add your own columns, click the Add custom column button. In the Lookup name field, type genre. In the Column heading field, type Genre (or 'iBooks Category', or whatever you please). In Column type, select Text, column shown in the tag browser. Click OK to exit the dialog. Restart calibre.
Populate your new custom column with the Genre/Category names to be used with iTunes/iBooks.
Open the Metadata plugboards dialog - Preferences|Import/Export|Metadata plugboards.
If you already have an existing plugboard for APPLE, select it in the Existing plugboards list, otherwise follow the steps described in the first example to create a new plugboard.
In the Source template field, add this template:
Code:
{#genre}
In the Destination field, select tags. This tells the templating engine "Retrieve the value stored in the custom column whose lookup name is 'genre', and apply it to the tags field."
Click the Save plugboard button, then Apply (upper left corner of the dialog) to save and close.
All books matching the specified formats sent to iTunes/iBooks will have this plugboard transform applied.
In the event that Preferences|Advanced|Plugins|Device Interface Plugins|Apple device interface|Customize plugin|Use Series as Category in iTunes/iBooks is checked, the normal behavior is to use the Series name as the Genre/Category in iTunes/iBooks. However, if a plugboard transform is defined for tags, it takes priority over using the Series name.
Display Authors as LN, FN in iTunes/iBooks
If you already have an existing plugboard for APPLE, select it in the Existing plugboards list, otherwise follow the steps described in the first example to create a new plugboard.
In the Source template field, add this template:
Code:
{author_sort}
In the Destination field, select authors. This tells the templating engine "Retrieve the authors_sort value, and apply it to the authors field."
Click the Save plugboard button, then Apply (upper left corner of the dialog) to save and close.
All books matching the specified formats sent to iTunes/iBooks will have this plugboard transform applied.
Force <dc:language> to specific value (ePubs only)
If you already have an existing plugboard for APPLE, select it in the Existing plugboards list, otherwise follow the steps described in the first example to create a new plugboard.
In the Source template field, add this template:
Code:
fr
In the Destination field, select language. This tells the templating engine "Set the value of <dc:language> to 'fr'."
Click the Save plugboard button, then Apply (upper left corner of the dialog) to save and close.
All books matching the specified formats sent to iTunes/iBooks will have this plugboard transform applied.
Preferences|General|Preferred output format to EPUB
Preferences|Conversion|Page Setup|Output profile to your iDevice.
Preferences|Plugins|Device Interface plugins|Apple device interface - click Enable/Disable plugin so that driver is enabled (not grayed out)
To send epubs or PDFs from calibre to your iDevice:
Select the book(s) to send from your calibre Library window, then click Send to device. Your books are sent to iTunes and iBooks.
To use Fetch News to download periodicals from the Internet:
Preferences|Interface|Automatically send downloaded news to ebook reader: Recommendation: checked
Preferences|Interface|Delete news from library when it is automatically sent to reader: Recommendation: checked
Click Fetch news, select the recipe(s) you want downloaded, click Download now then Save. When the Jobs spinner has stopped and your iDevice has finished syncing, you can read your downloaded periodicals as epubs in iBooks.
'Connect to iDevice' mode - more details
The iDevice driver is very different than a conventional calibre driver which sees the connected device as a USB drive. The iDevice driver accomplishes its tasks by talking to iTunes through an automation interface. As a result, settings in iTunes influence how various operations are carried out. I have not tested all of the possible combinations of iTunes settings that one might use, so it's entirely possible that your configuration will trigger unexpected results. This overview discusses the most common settings.
It's a good idea to sync your iDevice with iTunes before connecting your iDevice to calibre for the first time. Syncing apps and other media between your iDevice and iTunes before working with calibre will speed up transfers.
In iTunes Preferences|Advanced, the checkbox Copy files to iTunes Media folder when adding to library is significant. If the box is unchecked, iTunes remembers the location of the file (calibre's copy). If the box is checked, iTunes makes a copy of the file to its own storage. However, iTunes also modifies calibre's copy of the epub by adding a plist file. Files so modified by iTunes have been tested in the calibre viewer and in Adobe Digital Editions with no observed problems. Recommendation: unchecked.
If you are currently using iTunes to manage other media for your iDevice, and this is checked, leave it checked. iTunes will make its own copy of the added books rather than referencing calibre's copy. Any new media you add to iTunes will be stored in iTunes, as before.
In iTunes Device Summary, the Manually manage music and videos checkbox influences how syncs are accomplished:
If unchecked, calibre adds all of the transferred epubs to iTunes, then does a single sync operation to transfer all of the uploaded epubs to iBooks on your iDevice at once. If iTunes decides that other items need to be synced at the same time, it will do so during this sync cycle. This could include doing an incremental backup of your iDevice.
If checked, books are sent to the iDevice directly without a full sync cycle. (Calibre still keeps a copy into iTunes for future reference.)
There will be performance differences between the two settings depending on your usage patterns and your OS, but the driver is designed to work with either setting. Recommendation: start with unchecked. Windows users: things seem to work better with the checkbox unchecked.
Launch calibre. When a connected iDevice is detected, calibre will launch iTunes (minimized) if it's not already running. Calibre does not support multiple iDevices connected simultaneously. If calibre detects more than one physically connected iDevice in iTunes (iPad, iPhone, iPod Touch), the driver will not launch. This includes devices that have been ejected but not physically disconnected.
In calibre, set Preferences|General|Preferred output format to 'EPUB', and Preferences|Conversion|Page Setup|Output profile to 'iPad'.
The first time you connect your iDevice to calibre, calibre will be caching metadata and thumbnails as it scans iTunes and iBooks, making subsequent connections much faster.
The iDevice 'Reader' icon will be shown next to the 'Library' icon, indicating that calibre is connected to your iDevice. Click it to see books in the Books playlist on your iDevice.
To upload books in either epub or PDF format from calibre to your iDevice, select the book(s) to transfer from your Library, then click Send to device.
iTunes/iBooks supports a single Category tag. You may choose to use either a tag or a Series name as the Category by indicating your preference in Preferences|Plugins|Device Interface plugins|Apple device interface. Click Customize plugin. Use Series as Category in iTunes/iBooks has the following meaning:
checked: If your book has a Series name, it will be used as the Category in iTunes/iBooks. If your book does not have a Series name, a tag will be used.
unchecked: If you have multiple tags assigned to a book in calibre, the first tag beginning with an alpha character will be used. Special tags like '[Project Gutenberg]' or '+' are ignored.
Recipes and catalogs may be sent to the iDevice.
While calibre is running, it's best to do all iDevice-related actions through calibre rather than iTunes. Minimizing iTunes while calibre is running will reduce the temptatation to meddle.
Syncing is initiated after clicking Send to device or Remove books in the Reader view. In some cases, the syncing operation between iTunes and your iDevice will continue after calibre is done with its part. Watch the screen on the iDevice to know when the sync is done.
Unlike some other readers, the iDevice may be used while connected to the computer. This means that you can open your uploaded books in iBooks (after syncing is complete) while the iDevice is still connected to calibre. You don't need to eject the iDevice until you want to disconnect the iPad from your computer.
You can copy books from your iDevice to your computer using Save to disk, but be aware that all epubs acquired through the iBooks store are encrypted with Apple's Fairplay DRM, and thus not readable anywhere but on your iDevice.
Books with DRM are able to have their metadata updated in calibre.
iBooks stores epubs on the iDevice in uncompressed form. As a result, you will notice that the size of the book shown in calibre's Library window (and iTunes) is different than the size of the book on the iDevice.
What's the difference between 'Connect to iTunes' and 'Connect to iDevice'?
Spoiler:
'Connect to iTunes' uses iTunes as the Reader device. Clicking the Reader icon (the iTunes logo) shows a list of books in the iTunes Library. After sending books from calibre to iTunes, you sync iTunes with your iDevice at your convenience. This is the recommended mode.
'Connect to iDevice' uses your iDevice as the Reader device. Clicking the Reader icon (an iPad) shows a list of books in the iBooks library on your iDevice. Sending books from calibre to your iDevice happens immediately, without requiring you to sync. This mode is more complex.
Users are encouraged to use the 'Connect to iTunes' mode, as it adapts to the way that you're already using iTunes. You don't need to be concerned with any of the iTunes preferences or optional settings. When you're ready to update the contents of your iDevice, sync as you normally would.
If you choose to use 'Connect to iDevice' mode, spend some time reading and understanding the significance of the various iTunes settings that will affect calibre interactions.
Known Issues
Spoiler:
The driver currently sends comments from calibre to iTunes, however the automation interface to iTunes limits the transferred text to 256 characters. This is a limitation of the iTunes automation interface in both OSX and Windows.
If you use calibre to delete a book that you purchased through the iBookstore (free or paid), iTunes may pop up a dialog asking you to confirm. This does not happen with books transferred from calibre.
If you bring up a modal dialog in iTunes, for example Get Info to examine the book's metadata, the iDevice will appear to disconnect in calibre after a short time. As soon as you close the modal dialog in iTunes, the connection will be restored in calibre.
(OSX only) If iTunes|DEVICES|iDevice|Summary|Options|Manually manage music and videos is checked, the automation interface will generate an error when trying to send artwork to the iDevice. This is as of iTunes 9.2, hopefully that bug will be fixed by Apple at some point.
It is currently not possible to set the cover for PDFs via the automation interface. iTunes/iBooks uses the cover specified within the PDF file.
When transferring PDF files, there is a period of several seconds per book when it appears that nothing is happening. What is happening is that calibre is updating the PDF's metadata before sending the book, but there is no user feedback. There is an existing bug report #5949 addressing this issue. If you are sending multiple PDFs, the cumulative time without user feedback can be substantial.
'Connect to iDevice' mode can only deal with one connected iDevice. When the driver loads, it confirms that there is only one iDevice connected. However, if you connect a second iDevice later, calibre won't know which device to communicate with. Using 'Connect to iTunes' avoids this problem.
Under the hood
Spoiler:
The driver is tested with OSX 10.6 (Snow Leopard), Windows XP, Windows 7, and a 3G iPad. With OSX, 10.5 or newer is required (Leopard or Snow Leopard).
You must have iTunes installed on your system, as calibre talks to the iDevice through iTunes. You should install the latest version of iTunes, 9.2 as of this writing. If in doubt, download the most recent version from apple.com. To check your installed version, click iTunes|About iTunes in the iTunes menu.
There will be no Linux support in this driver, as there is no Linux version of iTunes.
The driver sends only epubs and PDFs to iBooks via iTunes via a physical USB connection.
If you have a cover assigned in calibre, it will be used in iTunes and in iBooks as the book's cover, superseding any cover within the file. As of iTunes 9.2 and iBooks 1.1, this works only for epubs, not PDFs. The cover art provided in the PDF will be used by iTunes/iBooks.
Calibre uses an automation interface to send commands to iTunes. The automation interface for OSX is completely different than the automation interface for Windows. Some operations have problems in one environment but not the other. The driver attempts to recover gracefully from any errors when talking to iTunes through the automation interface.
When downloading calibre recipes, the driver changes the metadata before sending the book to iTunes. For example, if you download the NYTimes, the title will be shown as 'The New York Times', and the author will be shown as today's date, e.g. 'Friday, June 18, 2010'. Title sort is 'New York Times', author sort is 'New York Times 2010-06-18'. Periodicals are tagged with 'News' by default, which becomes the Category for the book in iBooks.
If you are using recipes, be aware that Preferences|Interface|Automatically send downloaded news to ebook reader and Preferences|Interface|Delete news from library when it is automatically sent to reader should be set according to your preferences.
See Metadata Mapping and Plugboard Transformations section for current metadata mappings.
Metadata management: Preferences|Sending books to devices|Metadata management lists multiple options for how metadata is transferred to devices. The iTunes driver only transfers metadata when a book is sent to iTunes. The iTunes driver has no notion of collections. It doesn't matter what is selected in Metadata management - metadata is only transferred when books are sent to iTunes. If you want to update the metadata, send the book again.
If you have problems ...
Spoiler:
Books are disappearing from iTunes
If you have enabled/disabled the Apple device driver in Preferences, restart calibre.
If you are connecting multiple iDevices to calibre, use the 'Connect to iTunes' method described at the beginning of this post. Connecting directly to the device is an advanced user mode for single iDevices.
Calibre doesn't recognize my iDevice
Confirm that you have only one iDevice (iPad, iPhone, iPod) connected.
I don't want calibre to recognize my iDevice
If you would prefer that calibre ignore any connected iDevices, disable the Apple device interface in Preferences|Plugins|Device Interface plugins.
Calibre doesn't crash, but it's not doing what I expect
Confirm that you're up to date with the latest release of calibre.
If you're using 'Connect to iDevice', try using the recommended 'Connect to iTunes' mode as described at the beginning of this post.
Start calibre in debug mode. With iTunes running, open a command window then type:
Code:
calibre-debug -g >calibre.txt
Repeat the steps to recreate the scenario. Exit calibre, create a ticket in the calibre bug database. Attach 'calibre.txt' to your bug report. Please provide as much relevant context as possible to help recreate the problem, including the steps you took prior to observing the problem, your OS, and the nationality of your iTunes installation.
Calibre crashes
Confirm that you're up to date with the latest release.
If you're using 'Connect to iDevice', try using the recommended 'Connect to iTunes' mode as described at the beginning of this post.
If you're still crashing, launch calibre in debug mode. With iTunes running, launch calibre from a command window:
Code:
calibre-debug -g >calibre_crash.txt
Repeat the steps to create the problem. After the crash, create a ticket in the calibre bug database. Attach 'calibre_crash.txt' to your bug report. Please provide as much relevant context as possible to help recreate the problem, including your OS and the nationality of your iTunes installation.
Everything's perfect except for this one missing feature I can't believe you forgot
Please post comments, problem, suggestions regarding iDevice support in this thread. Include your OS, the version of iTunes you're using, and the nationality of your installed iTunes.
I get the Icon of iPad, but no books in list, and then this error.
'NoneType' object has no attribute 'Playlists'
Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 58, in run
File "site-packages\calibre\gui2\device.py", line 262, in _books
File "site-packages\calibre\devices\apple\driver.py", line 179, in books
File "site-packages\calibre\devices\apple\driver.py", line 1251, in _get_library_books
AttributeError: 'NoneType' object has no attribute 'Playlists'
In the graphic you posted, what is the name of your library? In other words, the topmost item in the left pane? And what is the name of the Books playlist? I can't quite read it.
I've submitted a revised driver to Kovid that will be included in the next release, hopefully it should resolve many of these early issues. The issue of letting the user specify the storage location for media will not be in the next release, as I'm discussing strategies with Kovid.
Part of the challenge is with iTunes being available in multiple languages, and two different automation grammars (OSX and Windows), so there may be a few more debugging iterations until the driver's stable. But, as is usually the case, it works great on my machine so I have some confidence it will become generally usable.
Using it on a Mac, I'm finding that when I hit 'send to device', this almost immediately triggers the book appearing in iTunes and a sync beginning that sends it to my iPad. The sync then ends and the iPad and iTunes are all ok. But Calibre is showing 'Send metadata to device' jobs still running that won't ever end and thus block all other Calibre job work. I have to quit Calibre completely and restart to get past this.
This is just FYI as I have read many posts where users have reported issues in transferring books to iPad and I think that the common factor there is Mac OS primarily.
I am using Win 7 (64bit) and Calibre 0.7 and my books were sent to my iPad without any hitch or delays.
06-05-2010, 06:37 AM
so I have some confidence it will become generally usable.
Similar Threads