Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 07-19-2012, 04:11 PM   #1
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
How I Manage eBooks with calibre

Revised 2015-07-30. Long post. Strategy, workflow, and layout — mapped in detail and example.


Approach
Spoiler:

Overview
  • Purpose. Describe my approach to managing ebooks.
  • Topics. Needs and Strategies, Workflow and Layout.

Needs and Strategies
  • Identify Needs. Identify current personal needs for ebooks. Reevaluate needs periodically and revise the list of needs to reflect any changes. Some of my needs are:

    • Feed the reading habit, primarily with fiction, or occasionally with nonfiction or news.
    • Avoid reading books with annoying formatting problems that adversely affect readability.
    • Maximize time for reading. Minimize time fixing formatting problems and maintaining book libraries.
    • Read comfortably during short or long reading sessions, usually at home or occasionally while out and about. Avoid eye strain caused by glare, brightness, or too-small screen. Avoid sore wrists from heavy device.
    • Find books quickly and easily regardless of actual location. Load and unload books quickly and easily on devices.

  • Devise Strategies. Devise strategies to meet needs, within the constraints of available ebook-related hardware and software technology. Review strategies periodically and revise them to reflect changes in needs or technology. Some of my strategies are:

    • Keep it simple. (See KISS principle, Wikipedia.)
    • Use calibre as the hub of my ebook activities. Supplement calibre with reading devices, reader apps, and other useful tools.
    • Process new books into calibre one author at a time, no more than a few books at a time, and finish processing them before adding other new books.
    • Keep metadata consistent by comparing with metadata on a good bibliographic site.
    • Choose content carefully. Be selective about which authors to read, their overall writing skill, and which publication types and subjects/genres fall within areas of interest. Do not collect books just because they are free or cheap.
    • Be discriminating about source and quality of formatting. Some sources tend to have formats with higher quality formatting and fewer problems than other sources. Track sources and use the better ones. Track format quality, fix formatting problems that are quickly fixable, and keep only formats that are readable with mild or no annoyance.
    • Standardize on EPUB as the format for most books. Make exceptions only for non-fiction PDF formats that are not separately available in AZW3 or EPUB, due to probable conversion problems and wasted time fixing them.
    • Read simply-formatted text-based books on e-ink device (most books, for me.) Read image-laden, color-formatted, or complexly-formatted books on a tablet (on rare occasions, for me.)
    • Gradually learn and improve ebook-related knowledge and skills. Note that different people do things differently due to different needs, strategies, skills, chosen technology, and personal inclinations.

Workflow and Layout
  • Refine Workflow. Derive workflow from needs and strategies, in conjunction with layout, within the constraints of available technologies and skill using them. Gradually refine each workflow, and keep it simple. I do three separate workflows, described in the sections Processing, Reading, and Maintaining.
  • Refine Layout. Gradually refine each layout and keep it simple, adapting to changes in needs, strategies, technology, skill, and workflow. Layout includes library structure and associated metadata scheme. A metadata scheme defines metadata elements for each column and conventions for their use. The Library section describes my library structure. The Metadata section defines my metadata scheme.


Processing
Spoiler:

Overview
  • Purpose. Transform downloaded ebooks into readable condition, accompanied by accurate metadata.
  • Topics. Browse Books, Add Books, Edit Metadata, Convert/Edit Formats.

Browse Books
  • Choose Current Author. Choose an author to become Current Author for the Processing workflow. Some ways to choose Current Author are:

    • Browse New Releases. Browse new releases and forthcoming books on a retail or bibliographic site. Fantastic Fiction has lists of books released or forthcoming for the current month, the prior 3 months, and the next 3 months. Internet Speculative Fiction Database (ISFDB) has lists of books released or forthcoming for the current month and the next several months. Amazon has lists of new books by genre. Browsing new releases provides an opportunity to investigate and potentially choose authors not yet in the library, or to update an author already in the library with newly available books. (See Links > Bibliographic Sites).
    • Browse Authors in Library. Check the author page of a bibliographic site for newly available books by an author. In a long-term library-update cycle, gradually work through the library, treating each successive author as the new Current Author, adding newly available books.
    • Other. Recommendations from family, friends, social networking sites, book reviews.

  • Search for Owned Books. Search for books by Current Author that are already owned. To limit the number of places to search, first consolidate books into as few places as practical. I keep all processed books in one main calibre library, so I search it for Current Author. The first two years of using calibre I also did an operating system search of a Pending folder containing books not yet added to calibre, until eventually all Pending books were added, one Current Author at a time.
  • Find List of All Books by Current Author. On a bibliographic site with consistent metadata for many authors, find a list of all books written by Current Author. I usually keep a browser tab open to Fantastic Fiction, and sometimes also Internet Speculative Fiction Database (ISFDB), pasting Current Author name into the relevant search box as necessary. Occasionally I open tabs for other helpful sites. For a list of potentially helpful sites, see the optional plugin Search the Internet.
  • Add Empty Books. To avoid duplication and repetition of investigative work in the future, add empty book wishlist placeholders for books to get in future, and add empty book exclusion placeholders for books not to get at all. An empty book is a book record with no associated book format. For each new empty book, enter metadata for Authors, Series, Title. Enter format status in custom column FormatQ (which stands for Format Quality): f0 for exclusion, or f1 for wishlist. For exclusion placeholders also edit Title to specify exclusion at author, series, or title level. (See Metadata > Custom Columns > FormatQ.)
  • Evaluate Current Author Status. If there are no books by Current Author to acquire now, go back to the first step in Processing > Browse Books to choose a different Current Author.
  • Note, Digital Rights Management (DRM). I assume in this workflow that DRM is not an issue. If DRM is an issue, see Apprentice Alf's Blog. I use the plugin.
  • Compare Sources. Optional. If desired, compare sources to determine where to acquire books to get now. In the Get Books feature in calibre, choose desired sources, then search for Current Author, sort by Store (source) to see all books there by that author, or sort by Title to compare prices between Stores (sources). I rarely bother to do this.

Add Books by Current Author, For Each Source
  • Choose Source. Choose a source. In general, favor sources that provide retail-quality ebook formats over sources that do not. I like Amazon for its conveniently large selection of titles. AZW3 format can be converted successfully to EPUB later. PDF is acceptable for complexly formatted nonfiction books (such as technical books, art books, or textbooks) only if good versions are not available in AZW3 or EPUB.
  • Download Books. For each book, download desired book format from this source. When I have the choice, I prefer downloading to computer rather than directly to device. For any books downloaded direct to device, save them from the device to the Downloads folder.
  • Standardize Filenames in Operating System. Standardize filenames in the operating system, into a structure compatible with the chosen regular expression for Adding Books. I use the structure "authors - series series_index - title"; series is optional; if there are coauthors I separate them with space ampersand space " & ". Manually editing each filename is not too time consuming because I add just a few books at a time. Otherwise I would use a file renamer tool to bulk rename a lot of files needing a similar filename structure changed to a different filename structure. (See Links > Filename Renamers; and Links > Filename Limitations).
  • Add Books. For each book in Download (or Pending) folder that has a preexisting wishlist placeholder, add the format by dragging it onto the Book Details pane of the wishlist placeholder. For other books in the Download (or Pending) folder, drag and drop them onto the Library View booklist to copy them into the library. My settings for adding books are:

    • Preferences > Adding Books. All options are unchecked on both tabs, except Copy to Library to preserve date added. Automerge is unchecked because I prefer each format of each book added as a separate book record. Read metadata from file contents is unchecked because the results are unpredictable, often requiring more work later to correct bad metadata or delete unwanted metadata. The only metadata I want set during the Add is Authors, Series, and Title. So I prefer reading metadata from filename using a regular expression.
    • Quick Preferences. The optional plugin Quick Preferences has a menu of regular expressions tailored for various filename structures. I use the Quick Preferences regular expression labeled "Author [- Series #]- Title", which is one of the defaults included in the plugin. This regular expression handles many variations correctly as long as the metadata in the filename is in the order author, series, title, with series being optional.
    • Note, Defaults. To see what the default settings are for any Preferences pane except Plugins, click the button Restore Defaults. Click the Don't Save button to return to prior settings, or the Apply button to keep the default settings.

  • Move Raw Files. For the books just Added, move the corresponding original files from Download (or Pending) folder to the relevant source folder in a RawBooks folder. Keeping these original downloaded raw files is a form of insurance, a one-to-one raw backup (not DRM-stripped, converted, or format-edited) of each book in the calibre library. The standardized and comprehensible filenames, edited previously to fit the regular expression for adding books, also will be useful in identifying any of these raw formats if necessary in future.
  • Enter Source. In calibre, enter source of format in Source. (See Metadata > Custom Columns > Source.)
  • Move Metadata for Authors, Series, Title. If necessary move metadata into the correct columns for Authors, Series, and Title. This is not necessary if the original files were appropriately renamed prior to the Add, unless there was an inadvertent typo.

Edit Metadata, For Each Newly Added Book by Current Author
  • Edit Authors, Author Sort, Series, Series Index, Title. For newly added books including placeholders, edit metadata as necessary for Authors, Series, Series Index, Title. If necessary, modify Author Sort using Manage Authors. Apply relevant conventions to the metadata. (See the Metadata section for details.) Authors and Title need to be clean prior to Metadata Download, because Metadata Download relies on Authors and Title for matching when there is not an ISBN in Identifiers.
  • Extract ISBN. Optional. If necessary to get specific metadata for a specific publication of a book, extract an ISBN using the plugin Extract ISBN, or in Edit Metadata Individually paste into Identifiers the ISBN of the desired publication copied from the bibliographic site or relevant publisher site. I rarely bother to do this.
  • Download Metadata. In Edit Metadata Individually, using metadata sources Amazon and Fantastic Fiction (an optional plugin), download metadata for the best choice of publication, fetching only Comments, Languages, Published, Publisher, Cover (automatic), and Identifiers (automatic). If dissatisfied with results of the metadata download, delete the Identifiers then download metadata again with different choice of publication or cover. Note: I do not download any other metadata because I previously set Authors, Series, and Title; I prefer relying on my own rating system rather than an outside Rating; and I would rather manually enter other desired metadata per my own metadata scheme faster than editing or deleting a downloaded mess of unpredictable, disorganized, or inaccurate metadata.
  • Edit Downloaded Metadata. While still in Edit Metadata Individually, if necessary edit the downloaded metadata: copy/paste better cover or Comments from a bibliographic site; trim cover manually using the Trim Image tool; edit Publisher, Published, Languages, Identifiers, Comments. (See the Metadata section for details.) Compare the metadata to the bibliographic site.
  • Enter and Edit Metadata. In the booklist, enter other metadata gleaned from information in Comments, compare with metadata on the bibliographic site, and edit as necessary for custom columns Types, Genre, Awards, and Read. If Title is not unique for different publications of one authors/title in this library, append edition or other publication-level metadata to Title in parentheses. Enter alternate author names in custom column aka, which stands for "also known as". (See the Metadata section for details.)

Convert/Edit Formats, For Each Newly Added Book by Current Author
  • Convert Format. Convert format to EPUB, unless it is a PDF. I standardize on EPUB as the format for most books and avoid adding PDFs to the library when possible, but would rather read a PDF as PDF than convert it to EPUB followed by wasting time fixing all the conversion problems. (See Links > Conversion.) The settings initially depicted in the conversion dialog box before the first conversion are the corresponding settings in Preferences; mine are:

    • Preferences > Behavior. Preferred output format: EPUB. Job priority: Normal. Both are calibre default settings. I chose EPUB as preferred output format for two reasons: Calibre's ebook viewer opens fastest with EPUB because it is an EPUB viewer; so to view a non-EPUB format calibre has to convert it to EPUB behind the scenes, which takes time. Also EPUB is an open standard, while AZW3 and other Amazon formats are proprietary.
    • Preferences > Input Options. All settings are calibre default.
    • Preferences > Common Options. Look & Feel, text justification: justify text. Look & Feel, Filter Style Information, Fonts: checked (which I uncheck only for specific conversion of nonfiction, magazines, comics). Page Setup, output profile: Kindle (initially set in Welcome Wizard dialog at first use of calibre, which I change to Tablet only for specific conversion of a format to be read on tablet). All other settings are calibre default.
    • Preferences > Output Options. All settings are calibre default.
    • Preferences > Tweaks. After gaining a few years experience with calibre, I set the tweaks save original file when converting/polishing from same format to same format: converting = False, and polishing = False. (Calibre defaults are both True.) Previously I had set converting True and polishing False. Saving the original format when converting supports repetitive conversions of original format to same format with different settings until getting it right, and also serves as a backup in case of fatal mistakes while editing the format, to allow restoring the format from original by right clicking the original format in Book Details or Edit Metadata Individually, then choosing "Restore 'format_name' from the original". Now that seems redundant because I rarely convert more than once, and I keep separately the original raw format that was copied into the calibre library; if necessary the raw original can replace a format. Polish Books does not do a conversion and its limited options are not likely to be destructive, so previously I had set the polishing tweak to False.

  • Assess Format. Open the format in the viewer or editor most appropriate for that format, such as calibre Viewer for EPUB, or Adobe Reader for PDF. I prefer using calibre Edit Book for EPUB, or Preview (on OS X) for PDF. Scroll or page through the format, looking for formatting problems that interfere with readability or are annoying. Initial setup: in Preferences > Behavior > "Use Internal Viewer," check any format to be opened automatically by calibre Viewer, and uncheck any format to be opened automatically by an external application per the operating system setting for that file extension; I use the calibre defaults.
  • Edit Format. If EPUB, adjust styles as desired and fix any formatting problems that adversely affect readability and are quickly fixable. Then assess the results, and if necessary edit the format again, better, then reassess. If PDF, do not bother to waste time fixing problems: this PDF either will be replaced in future with a good EPUB, or the PDF will only occasionally be used for reference. (See Links > Format Editing, and Links > Conversion.) Use one of these approaches for editing the format:

    • Fix Underlying Code in Editor. In this approach, directly edit the underlying code in the format. This requires basic knowledge and skills with HyperText Markup Language (HTML), eXtensible HyperText Markup Language (XHTML), and Cascading Style Sheets (CSS). CSS simplifies applying styles to text. HTML marks up text for construction of web pages. XHTML, largely based on HTML and extending it, also marks up text. EPUB uses XHTML. AZW3 wraps EPUB. MOBI uses HTML-like tags. Tools for fixing the underlying code are:

      • Edit Book (EPUB, AZW3). Select Book, open calibre's Edit Book feature, edit HTML/XHTML and CSS files, save. I use this tool because it is easiest and most convenient. I usually edit only a few styles in the CSS.
      • Sigil (EPUB, HTML, TXT). Open in Sigil, edit HTML/XHTML and CSS files, save EPUB. Sigil is an open-source EPUB editor.
      • Text Editor (EPUB, AZW3). Unpack the format, use a text editor to edit the HTML/XHTML and CSS files, repack the format. Useful adjunct tools include calibre Unpack Book (EPUB or AZW3), and the optional plugin KindleUnpack (AZW3, MOBI, other Amazon formats). See partial list of code-oriented text editors in Links > HTML/XHTML.

    • Fix in Word Processor. In this approach, edit in a word processor, with calibre automatically dealing with the underlying code during conversion. This requires no knowledge/skill with HTML/XHTML and CSS. I used this blunt-force approach with Word until I learned enough CSS and XHTML to use Edit Book in calibre. The workflow steps are:

      • Convert to RTF or DOCX. Convert EPUB to RTF or DOCX. (Note: I tested this with RTF, but not with DOCX.)
      • Save RTF or DOCX to Disk. Save the RTF or DOCX to disk into a Fix Formats folder. My preference settings for Saving Books to Disk include: update metadata in saved copies, convert non-English characters, show files in file browser after saving to disk, save template:
        {authors}{series:| - | }{series_index:0>2s}{title:| - |}{#formatq:| (|)}
      • Edit RTF. Open the saved RTF or DOCX copy in a word processor, either Microsoft Word, OpenOffice Writer, or LibreOffice Writer. If markup characters are not visible, in the View menu turn on markup (Word) or nonprinting characters (Writer). Fix various annoying formatting problems using find/replace.
      • Anchor Images. In Writer, if there are images, anchor each image to a paragraph. This is not necessary in Word.
      • Save As. From Word save as DOCX, or from Writer save as ODT, into the Fix Formats folder. Do not save directly into the calibre library folder because calibre will not detect it, which is one reason why the format was saved to disk from calibre into the Fix Formats folder in the first place.
      • Add Format. Add the DOCX or ODT format to selected book record in calibre, by dragging from Fix Formats folder and dropping onto Book Details.
      • Convert to Preferred Format. Convert DOCX or ODT to EPUB.

  • Fix Other Minor Problems. Optional. If necessary, fix minor formatting problems that adversely affect readability and are quickly fixable. Then assess results, and if necessary fix minor formatting problems again, better, then reassess. Some useful tools are:

    • Convert. If formatting problems seem amenable to fix through conversion, in the Conversion dialog box tailor a conversion with settings that will fix as many of the formatting problems as possible, reconvert, assess the results. If necessary repeat with one or more different settings then reassess.
    • Edit ToC. Fix problems with the Table of Contents (ToC) using the calibre feature Edit ToC. I usually skip this. I never use ToC when reading fiction, and the little nonfiction I read usually already has a good ToC.
    • Quality Check and Modify ePub. Identify specific problems in the format with the plugin Quality Check, and fix many of the them with the plugin Modify ePub (EPUB only), without doing a conversion. Quality Check also can do a few small fixes to metadata. If necessary, which is rare, I use one or the other plugin to identify or fix a specific problem.

  • Balance Between Extremes. As an ebook consumer primarily reading for enjoyment, my objective is to spend the least amount of time fixing formatting problems to make a book readable with no annoyance. I ignore formatting problems that do not affect readability, ignore formatting problems that are only mildly annoying but are not quickly fixable, and fix formatting problems that are annoying and quickly fixable. If that does not achieve readability with mild or no annoyance, I remove the format and make the book a wishlist placeholder. That is how I balance between ignoring all formatting problems, or striving for perfect formatting for every book. See Thoughts for those new to ebook reading, post by jackie_w.
  • Enter Format Quality Rating. Enter the format quality rating and any other relevant format-level tags in FormatQ, which stands for Format Quality. (See Metadata > Custom Columns > FormatQ.)
  • Remove Unnecessary Formats. Remove all formats in wishlist or exclusion placeholder records. For other book records, remove all unnecessary formats such as ORIGINAL_EPUB, AZW3, MOBI, DOCX, or ODT, retaining just one format in the book record, the master format. For me, the master format is either an EPUB or, when unavoidable for nonfiction, a PDF.
  • Remove Duplicates. If the book is one of a set of duplicate book records, compare metadata between the duplicates. Correct metadata if necessary for the duplicate containing the format that has the best format quality rating, keep that record, and remove the other duplicate record(s).
  • Count Pages. If EPUB, use the plugin Count Pages to enter page count in custom column Pages. If PDF, open, look for last page number, and enter that number in Pages. (See Metadata > Custom Columns > Pages.)
  • Polish Book. If EPUB, polish the format in the library. In the calibre feature Polish Books, I check only the options to update metadata and update cover in the format. Depending on type of format, in conversion or export operations calibre automatically updates the cover and metadata in the conversion output format or the exported copy, but not in the conversion input format or the format copied for the export. Polish Books (EPUB or AZW3 only), Embed Metadata (various formats), and the plugin Modify ePub (EPUB only) all operate on relevant format(s) in the library without doing a full conversion. If the format is PDF, I do not bother to embed metadata.


Reading
Spoiler:

Overview
  • Purpose. Read books on reading devices.
  • Topics. Set Up for Loading, Load Device, Read Books, Revise Metadata, Set Up Device.


Set Up for Loading
  • Set Up New Device. If loading to a new device that has not yet been set up with calibre, see "Set Up Device" at the end of this section. Before buying a new device, consider size, weight, resolution, glare, color versus black and white, capability to handle preferred format, and general suitability per needs, strategies, inclinations, habits, types of preferred content. Generally for me, smartphone is too small, large tablet too heavy, 6 or 7 inch screen okay. I read most fiction or popular nonfiction on e-ink device (PRS-T2). I occasionally read technical, image-laden, or color-formatted content on small tablet (Nexus 7). When necessary (rarely) I read large-display content such as glossy magazines or comic books on computer or large tablet (old iPad).
  • Set Up a Way to Find Books to Load. This is an initial one-time action, supplemented by ongoing tagging. I use a saved search "To Load" to find books with metadata elements indicating "to read", "catalog", or "news." At any time, such as while processing books or browsing the library, for any books I want to read soon, I enter the tag for "to read" in custom column Read. Alternatively, some users rely on the plugin Reading List to manage reading lists and sync specific lists to specific devices.
  • Fetch News. Optional. In the Fetch News feature, initially schedule automatic News downloads, or use one of the manual Download buttons at any time. Calibre automatically converts news downloads to preferred output format and enters the tag "News" in Tags. For news downloads, I do not assign any other tags or fix formatting problems. Occasionally I use the News feature manually: select a source, download now, glance through the resulting book in Viewer, and if something looks interesting, leave the book in the library so I can load it on device later from the results list of the saved search "To Load" which includes news downloads. In the Fetch News dialog, I set "delete downloaded news older than" to 30 days. In Preferences > Behavior, I unchecked "automatically send downloaded news to ebook reader" and unchecked "delete news from library when it is automatically sent to reader."
  • Create Catalog in Book Form. Optional. Create a catalog in book form that lists all the books in the library, prior to loading devices. Calibre automatically assigns the tag "Catalog" in the Tags column. My saved search "To Load" automatically includes that. I set the catalog to show different symbols for wishlist and exclusion placeholders. See calibre User Manual, Creating AZW3, EPUB, MOBI Catalogs.
  • Choose Device and Loading Method. Choose the device to load, and choose a loading method. See calibre User Manual, Frequently Asked Questions, Device Integration. Loading methods include:

    • Calibre Companion. Send (push) from calibre through wireless connection between calibre on computer and Calibre Companion on device. Calibre Companion is currently available only for Android devices. I like this method because it is simple, does not require a USB cable, allows loading books onto device in bulk from calibre, and also allows removing books in bulk from device through calibre.
    • USB Cable. Send (push) from calibre through USB cable to device. I like this method because it allows bulk loading from calibre, and also allows removing books in bulk from device through calibre.
    • Email. Send (push) from calibre through email to device. For various smartphones or tablets, books can be emailed to an email account on the device, then opened in a reading application. For Kindle, in calibre Preferences > Sharing > Sharing Books by email, amend the email addresses listed on Amazon > Manage Your Kindle > Personal Documents, add the word "free" after the kindle name and @ sign: "free.kindle.com." Calibre automatically includes a delay between emailing each book of a bulk selection, per the tweak in Preferences > Tweaks > "The number of seconds to wait before sending emails." I avoid the email method because the device must be connected to remove books in bulk from device through calibre, and it is simpler and faster to load in bulk while the device is connected, without the forced smtp delay.
    • App on Device, from Server. Pull wirelessly by web browser or reader app on device, using a catalog on server. Requires first generating an online catalog with calibre content server or an alternative catalog-on-server tool such as Calibre-PHP Content Server, calibre2opds, or Calibre OPDS PHP Server. A web browser downloads the selected format then usually pops up a dialog box for choosing which reading application to open the book; this works on any device with a web browser. Or a reading app downloads the selected format directly; this works for tablet and smartphone reading apps with capability to access an online OPDS catalog. I usually avoid this loading method because it does not allow loading in one bulk operation, and the device must be connected anyway to remove books in bulk from device through calibre. For the rare occasions I do use the content server, in Preferences > Sharing > Sharing Over the Net my non-default settings are server port 7070, username L1, password: ••••••••. (See Links > Catalog on Server.)
    • Cloud Syncing Service. With Dropbox (or substitute a different cloud syncing service as desired): Pull wirelessly from a calibre library stored in a Dropbox folder on the computer by selecting a book link in the Dropbox folder on device, then selecting a reading application from the Dropbox popup dialog box. This presumably works on all devices on which Dropbox is installed, with various reading applications that can read a book with that file extension. I avoid this method because it does not allow loading in one bulk operation, the device must be connected anyway to remove books in bulk from device through calibre, and in general I minimize my use of clouds.

Load Device (For Each Device to be Loaded)
  • Connect Device. Ensure no other device is connected to the computer. Then connect wirelessly to load an Android device such as Nexus 7, or connect with USB cable to load other devices such as Sony PRS-T2, Kindle Touch, Kindle Keyboard, Apple iPad, or Apple iPhone.

    • Android. Ensure the wireless connection is started in calibre, in the Connect/Share menu, "Start or Stop wireless device connection." Turn on device. Open Calibre Companion on device and click the Connect button to start the wireless connection to calibre.
    • Other Device. Turn on device, and connect USB cable to computer and device. If iOS device, open relevant application per [Device Plugin] iOS Reader Applications.

  • Revise Metadata for Previously Read Books. For any previously read books still on the device, do the steps in the "Revise Metadata" subsection below if they were not done shortly after reading the books.
  • Remove from Device Previously Read Books. In calibre Library View select and remove from device the previously read books.
  • Select Books to Load. In calibre Library View, choose the Saved Search "To Load" to bring up a list of books to load to devices. Select books to load for this device, according to the type of reading material or formatting best suited for this device. I prefer reading simply-formatted text-based books on an e-ink device. I prefer reading image-laden, color-formatted, or complexly-formatted books on a tablet.
  • Send Books to Device. In calibre Library View, send selected books to device.
  • Eject and Disconnect. In calibre, eject device. If connected by USB cable, disconnect it. If connected wirelessly through Calibre Companion, Calibre Companion automatically disconnects the wireless connection.


Read Books
  • Read Book(s). Read book(s) on device(s).


Revise Metadata, For Each Book Read
  • Edit Read. Soon after reading each book, in calibre Library View enter the content rating in Read and remove the metadata element meaning To Read. (See Metadata > Custom Columns > Read.)
  • Edit Format Quality Rating. If any formatting problems were discovered while reading, fix any that are more than mildly annoying right away, or downgrade the format quality rating in FormatQ. (See Metadata > Custom Columns > FormatQ.)
  • Revise Other Metadata. Revise any other relevant metadata, such as Genre, to reflect newly gained knowledge about the book. (See Metadata section for details.)
  • Remove Throwaway Format. If AZW3, MOBI, or other format was generated from EPUB to read on Kindle, in the calibre library retain the EPUB master and remove the AZW3 or MOBI format. Some calibre users prefer to keep such extra formats, rather than remove them. I remove them. Generally I will not reread a book again for at least two years. By then I will probably have newer and better reading devices. If or when necessary, with calibre I will convert-on-demand a fresh device-specific format from the EPUB master.


Set Up Device (Initial One-Time Tasks)
  • Review General Device Settings. Optional.

    • Sending Books to Devices. These settings apply to devices in general. Metadata management: Automatic management, which syncs metadata in calibre library and metadata on device at every connect. A save template constructs the folders and filename for storing books on any device; this generic save template can be overridden for a specific device by customizing the save template in the Device Interface plugin for that device, which I have not needed to do. I limit author_sort to the first author_sort name, and shorten series. Title is automatically truncated when necessary, so there is no need to shorten title in the template. My save template for Sending Books to Devices is:
      {author_sort:sublist(0,1,&)}{series:shorten(3,-,16)| - | }{series_index:0>2s}{title:| - |}
    • Collections, Bookmarks, Annotations, Syncing. I manage books from within calibre and I load devices temporarily a few books at a time, so I do not need or use collections to organize books on devices. (See Metadata > Calibre Columns > Collections.) Also I do not use bookmarks in calibre or annotations in calibre or on device. The only syncing I want is calibre's syncing of metadata per "Automatic Management" on any device, Calibre Companion's syncing of metadata on connected Android device, and the device keeping track of my reading position until I finish a book. Setup for collections, bookmarks, or annotations is usually different for different types of devices, involving different plugins, custom columns, or settings, which I avoid simply because I have no need to use those features.

  • Customize Relevant Device Interface Plugin. To customize a Device Interface plugin, in Preferences > Plugins > Device Interface Plugins select the plugin and click Customize Plugin button, or when the device is connected choose Configure this Device under the Location Manager icon. Generally all my device settings are default, except any settings for collections. For each device interface, I use the same save template as in Sending Books to Devices; and if the option is available "use sub directories" is checked. I use these Device Interface plugins:

  • Create Metadata Plugboard(s) for Device. When a format is sent to a device or folder, the templates in the metadata plugboard for that device and format put metadata into the relevant destination fields inside the format. The metadata displayed on the device is usually read from the fields in the format. My metadata plugboards are:

    • Android Devices. Works with Calibre Companion. Format: any format. Device: SMART_DEVICE_APP.

      • Destination field authors:
        {author_sort:sublist(0,1,&)}
      • Destination field title:
        {series:|| }{series_index:0>2s}{title:| - |}

    • Apple iOS Devices. Works with optional device interface plugin: iOS Reader Applications. Format: any format. Device: APPLE.

      • Destination field authors:
        {author_sort:sublist(0,1,&)}
      • Destination field title:
        {series:|| }{series_index:0>2s}{title:| - |}

    • Kindle. Format: any format. Device: KINDLE2.

      • Destination field authors:
        {author_sort:sublist(0,1,&)}
      • Destination field title:
        {series:shorten(3,-,8)|| }{series_index:0>2s}{title:| - |}

    • Sony, Plugboard 1. Format: device_db. Device: PRST1. See (Sony)…How do I set it up..., thread Post #15, by jackie_w.

      • Destination field authors:
        {author_sort:sublist(0,1,&)}{series:shorten(3,-,16)| - | }{series_index:0>2s}{#genres:shorten(3,,0)| - |}

    • Sony, Plugboard 2. Format: any format. Device: PRST1.

      • Destination field authors:
        {author_sort:sublist(0,1,&)}{series:shorten(3,-,16)| - | }{series_index:0>2s}{#genres:shorten(3,,0)| - |}


Maintaining
Spoiler:

Overview
  • Purpose. Improve performance and minimize maintenance issues.
  • Topics. Maintain Libraries, Tune Performance, Update Software.

Maintain Libraries, For Each Library
  • Beware Making Direct Changes Inside Library Folders (Ongoing). In the operating system avoid making direct changes to the calibre Library folders, subfolders, and files. Make any desired changes through the calibre Graphical User Interface or the calibre Command Line Interface. Make exceptions to this rule only with beforehand knowledge of the potential dangers. For further discussion, see Want to change the folder structure of the Calibre library?, sticky by Manichean & Worldwalker.
  • Backup Library and Configuration Folders (Frequently). Regularly backup the calibre configuration folder and each library folder. On OS X, I use Time Machine for automatic hourly backup of my Boot drive, which contains the calibre Configuration folder and Library folder, to an external drive. I also set Carbon Copy Cloner to automatically clone daily the Boot drive to a second external drive. Note the calibre configuration folder can be found by clicking the button Open calibre configuration directory in Preferences > Miscellaneous.
  • Check Library (Annually). On the Library icon drop-down menu, select the Library Maintenance command Check Library. Check if the files in library match information in the database. If calibre reports any problems, look in the operating system in the calibre library folder at each problem to ensure each problem is okay to delete, but do not change or delete files there. To delete any problems, in the Check Library dialog box put a check in checkboxes for the problem files to mark them, click the Delete Marked button. Or click the Fix Marked button when it is available for any problems that calibre can fix automatically.
  • Clean Up Metadata (Annually). In all columns with text, clean up minor variations (including typos) in metadata elements. This improves accuracy of searches.
  • Update Awards (Annually). Once a year, review awards for best novel on Wikipedia, and update the library for awards. (See Metadata > Custom Columns > Awards.)
  • Update Authors (Ongoing). Gradually during a long-term cycle, update each author with newly available books.

Tune Performance
  • Set Antivirus Auto-Scan Exclusions (Once, and After Library Moves). Initially, set the security/antivirus application to auto-scan, but exclude calibre Library folders from scans. After moving a library, reset the exclusion for that library; I usually forget to do that until the slower performance causes me to remember. When auto-scan is on, books are scanned for malware such as viruses at download time so there is no need to scan them again whenever calibre accesses book files, which can cause significant slow-downs in performance because calibre accesses a lot of files during the normal course of business.
  • Adjust Performance Settings (Once on Each Computer). Job priority (Preferences > Behavior) controls the speed and resource use of calibre worker processes; I keep this at the highest setting, Normal. Maximum simultaneous conversion/news download jobs (Preferences > Advanced > Miscellaneous) limits the number of CPU intensive background tasks; I set this to 6. Also I checked the option to Limit the maximum simultaneous jobs to available CPU cores (Preferences > Advanced > Miscellaneous). I chose these performance settings based on testing CPU-intensive jobs such as bulk conversion on a computer with an Intel i7 quad-core processor, while leaving some capacity for the processes of other open applications.
  • Manage Composite Columns and Templates (Once). In Preferences > Add your own columns, remove unnecessary composite columns (custom columns built from other columns with templates). For remaining composite columns uncheck the checkbox Show in tag browser. If displaying formats in a composite column in the booklist, use the template {:'approximate_formats()'} rather than the template {formats}. Translate any complex template programming to General Program Mode, and ensure the tweak compile_gpm_templates is set to true.
  • Manage Column Coloring and Icons (Once). Limit or remove rules for column coloring and column icons in large libraries. For me so far the usefulness of column coloring outweighs the slight delay in library startup time.
  • Limit Searched Metadata (Once and Ongoing). In general, use column lookup names in all search criteria in the search bar, saved searches, and virtual libraries. In Preferences > Searching, turn on Limit the searched metadata, and enter just a few fields for non-prefixed searches, ensuring Comments is not one of them, to speed up search results when making a mistake typing the column lookup name or forgetting to use it. I limit non-prefixed searches to Authors and #aka.
  • See. See for further information: Calibre Speedup (ways to fine-tune calibre performance), thread by Adoby.

Update Software
  • Update calibre. Updates of calibre are usually released every Friday, sometimes every other Friday. I like to install new versions right away, though it is not necessary so frequently.
  • Update Plugins. Updates of optional plugins that are available through calibre can be installed with a few clicks from within calibre. For other optional plugins, periodically check the relevant plugin site and install any updates manually.
  • Update Other Software. To minimize the frequency of maintenance or performance issues, for computer and devices keep operating systems and other software such as editors, reader applications, and security software up to date. Also keep firmware up to date for all devices. So far I have had no need to jailbreak any devices, but if I did I would watch for and update software and firmware for each jailbroken device.


Library
Spoiler:

Overview
  • Purpose. A calibre library ideally should support needs, strategies, workflow, and metadata scheme. This section describes library structure.
  • Topics. General Guidelines, Book Details, Tag Browser, Searching, Sorting, Columns, Order of Columns.

General Guidelines
  • Terms. Metadata is data about data. In a scenario of managing books, the books are the data, metadata describes the books, and a metadata scheme organizes and describes the metadata in a way that facilitates managing and using books. A metadata scheme includes definitions for each column, definitions for each metadata element in each column, and conventions for using them. A library houses the metadata scheme in its column structure. A metadata element is a member of a set of metadata elements that can potentially live in a column that lives in the library. A metadata element also applies to one or more specific books. Each book is a row in the calibre library view as well as a subfolder with files in the calibre library folder. For related terms and concepts, see Links > Metadata Terms.
  • Number of Libraries. Limit the number of libraries to the fewest necessary. There are pros and cons to using one general library for various purposes, or multiple libraries each for a more specific purpose. The question is whether one library structure with corresponding metadata scheme is sufficient to meet all needs or not.

    • Multiple Libraries. Separate libraries could be useful in various scenarios, for example: adult library versus child library, shared library versus unshared library, any number of specific libraries each tailored for a specific publication type or media type with specialized custom columns, or one former large library split into two smaller libraries to improve calibre performance. Each extra library requires additional work in setup and maintenance.
    • One Multipurpose Library. Many calibre commands apply only selected books within the currently open library. Searching metadata in one large library is simpler and faster than searching sequentially in multiple libraries. Copying/pasting metadata or comparing books within one library is faster and simpler than across multiple libraries. The content server serves only the library from which it was started. Maintenance is faster in one large library as opposed to multiple libraries.
    • My Library. I use one multipurpose calibre book library, named L1. I process and store all books in L1. I add no more than a few books at a time to L1, then process them there immediately before adding any other new books.

  • Library-Specific Structure. Calibre applies library-specific settings when starting up a calibre library. Each calibre library has its own library structure. When creating a new empty library with the option checked to copy structure from current library, all library-specific settings are transferred to the new library. When creating a new empty library with the option not checked to copy structure, calibre transfers only the default predefined columns as well as other default settings for the various aspects of library structure. For each library, consider:

    • Amount of Metadata. Limit the number of columns to the fewest necessary. The more there are, the more time necessary for processing each book and maintaining each library. Do not use a calibre column or custom column unless it meets a specific need that otherwise cannot be met as easily.
    • Distinct Attribute. With the exception of Tags, each default calibre column targets a different distinct attribute of books. Ensure each added custom column does that too, with a unique column name identifying the targeted attribute. Ensure that there is a good reason to keep local metadata for each attribute, and that this metadata somehow facilitates the main purpose of the library. Ensure the metadata elements in each column specifically represent the distinct attribute (except possibly the column Tags). For example, in a column targeting the attribute "publication type" each metadata element represents a specific publication type (collection, omnibus, anthology, short story, and so on); or in a column targeting the attribute "award" each metadata element represents a specific award (Hugo for novel, Nebula for novel, Campbell for novel, and so on).
    • Unique Metadata Elements. Limit the number of metadata elements per column to the fewest necessary. Ensure the name of each metadata element is unique within its column.
    • Metadata Element Abbreviations. If using abbreviations for metadata elements, ensure each abbreviation is easy to remember and provides a clue to its meaning. Easily memorized abbreviations, sounding indicative of the full word or phrase when spoken aloud, are almost as human-readable, and often more immediately human-readable than the full word or phrase. Particularly for "comma separated text" columns, on a small computer screen displaying a library booklist with limited column widths, it is not always possible to see at a glance unabbreviated metadata off to the right of the visible portion without doing an extra step of scrolling within the cell, or hovering the cursor to see the tooltip. Abbreviations are convenient for seeing at a glance more metadata in a smaller horizontal space. On the other hand, the more people using a library containing abbreviated metadata, the more potential confusion.

Book Details
  • Book Details Pane. The contents of the Book Details pane is controlled by settings in Preferences > Look and Feel > Book Details. Set the location for Book Details (not library-specific) in Preferences > Look and Feel > Main Interface > User interface layout: Narrow for below booklist, or Wide for right of booklist.
  • My Book Details. In my library L1, this metadata is shown on the Book Details pane: Authors, Series, Title, Identifiers, Publisher, Date, #Genres, and #FormatQ, as well as the cover. I maximize horizontal space for seeing columns in the booklist by locating Book Details below the booklist. I do not use Roman numerals for series, or show cover size. I replaced the "default author link template" for Wikipedia with the Fantastic Fiction author link template copied from Search the Internet plugin:
    http://www.fantasticfiction.co.uk/search/?searchfor=author&keywords={author}

Tag Browser
  • Categories. Library structure includes settings for Tag Browser categories. I hide the Tag Browser categories Identifiers, Languages, News, Rating, Tags, and Searches. I do not have any additional custom categories defined, aside from the categories of custom columns automatically shown in the Tag Browser. In Preferences > Look and Feel > Tag Browser, "tag browser category partitioning method" is disabled, and "show average ratings in tag browser" is unchecked. I manage searches from the search buttons to the right of the saved search menu box in the search bar.
  • Hierarchical Categories. Library structure includes settings for whether or not a category is hierarchical. I use hierarchical metadata for Series, but I do not need to see Series in a hierarchical breakdown in the Tag Browser per Preferences > Look and Feel > Tag Browser > Categories with hierarchical items, which I keep set to the default "none", so I use semicolon space "; " as subgroup separator in Series because it looks better than the period "." separator.
  • Tag Browser Searches. Using the Quick Search box in the search bar is faster and more flexible than using the Tag Browser for searches. Taking the time to learn the search syntax was well worth it. I rarely do searches from the Tag Browser.
  • Managing Metadata Elements. I use the Tag Browser to manage metadata elements when there are a lot of them to rename or delete, usually in conjunction with restructuring columns or metadata elements. The plugin Find Duplicates has a nice feature for finding and editing metadata variations in Authors, Series, Publisher, and Tags, but I do not use it, as I prefer to manually edit metadata elements in those columns and others in the Tag Browser. Clean, standardized, non-varying metadata elements are easier to remember when entering searches, and enable more accurate search results. They also look better in Booklist and Book Details.

Searching
  • Quick Search. For searches I use the search bar above the booklist, rather than the Tag Browser or the Advanced Search dialog. I prefix most search terms with column lookup name — or with a grouped search term which uses column lookup name(s) — to get results faster than without lookup names. I limit non-prefixed searches to only authors and #aka (also known as); this allows typing into the quick search box just author first name, last name, or whole name without column lookup name or quotes, getting results as quickly as if the lookup names were used. See calibre User Manual, Search Interface. Example quick searches:
Code:
laurell k
series:blake
series:"anita blake"
title:"guilty pleasures"
#source:amazon
hamilton
  • Grouped Search Terms. Grouped search terms are an aspect of library structure set in Preferences > Searching. They are convenient for combining more than one column into one search term, or as shortcut aliases for lookup names of single columns. Enter the term name and column(s) for a new grouped search term then Save it and Apply, even if the desired column is not represented in the drop-down list, then restart calibre before trying to use the new term. Below are the grouped search terms I use in library L1, each with an example "contains" search.
Code:
Term   Column Lookup Names(s)     Example Search   Explanation
----   ------------------------   --------------   ------------------------
a      authors, #aka              a:gaiman         Gaiman in either column
c      comments                   c:assassin
f      formats                    f:pdf
s      series                     s:"rogue angel"  
t      title                      t:mythago

aw     #awards                    aw:h             Hugo award winner
fq     #formatq                   fq:f1            wishlist, format quality
g      #genres                    g:fn             fantasy
pp     #pages                     pp:<150          potential shorts
r      #read                      r:TR             books To Read
so     #source                    so:MR            books from MobileRead     
ty     #types                     ty:om            omnibuses

zz     title, series,             zz:fantasy       fantasy in any of these
        comments, #genres          or zz:fn         columns
  • Virtual Libraries. Virtual libraries are an aspect of library structure employing search criteria to partition sets of books within one library. I use virtual libraries for Fiction and Nonfiction in library L1, based on saved searches with the same names. See calibre User Manual, Virtual Libraries.

  • Saved Searches. Saved searches are an aspect of library structure useful as the basis of virtual libraries, or for quick reuse of frequent searches. See calibre User Manual, Saving Searches. Some of the saved searches in library L1 are:
Code:
Search Name       Search Criteria                          Explanation
---------------   -------------------------------------    -----------------
To Load           #read:TR or tags:Catalog or tags:News    TR is "To Read"
Format Problems   not (#formatq:f4 or #formatq:f5)         f4 good, f5 great
NewBooks 10days   date:>=10daysago                         within timeframe
Authors > 1       authors:#>1                              coauthors exist
calibre-auto      tags:news or tags:catalog                automatic tags
Fiction           not search:"=NonFiction"
Nonfiction        #types:nf or tags:news or tags:catalog

Sorting
  • Sort History. Calibre saves column sort order and associated column sort direction in a sort history for each library.
  • Sort-Related Tweaks. Tweaks are not library-specific structure, but are configuration settings applied to all locally known libraries. Sort-related tweaks need to be considered because they may affect sort order of book records, or the number or order of sorted columns. In Preferences > Tweaks, search for sort to find them. For example, I set dates to be sorted using visible fields, and set title and series to be sorted strictly-alphabetic rather than library-order.
  • Sort Order and Keyboard Shortcuts. Keyboard shortcuts are another type of configuration applying to all locally known libraries. A Keyboard Shortcut set with the plugin View Manager quickly resets sort order to a desired pattern, and moves a selected newly added book from the top of the booklist to its sort position, centered in the booklist display. I use View Manager keyboard shortcuts frequently. A small price to pay for that convenience is having to manually sync the two views in View Manager with calibre's column order for a newly added column or a change in column order. The keyboard shortcuts I set in View Manager are:

    • Preferred Sort Order. The keyboard shortcut Command 1 resets to the view "Main TSA - Cmd 1" which is my static column order and preferred sort order: Title within Series within Authors, all ascending.
    • Alternate Sort Order. The keyboard shortcut Command 2 resets to the view "Main TS - Cmd 2" which is my static column order and alternate sort order for working with varied-author series: Title within Series, both ascending.

Columns
  • Calibre Columns in Library View. Some of these default calibre columns in Library View of my library L1 also have associated settings that are not default. See Metadata section for description of each column.
Code:
Calibre Columns                 S  Shown             Visibility
in Library View                 H  Hidden            ----------
                                -  not applicable    Tag browser   E
                                *  See description   Book details  d  Groupd
--------------------------------------------------   Library view  i  Search
                                         Separator   Edit dialog   t   Term/
Name          Lookup         Type        or Format   T  B  L  E    ?   Alias
-----------   ------------   ---------   ---------   ----------   ---   ----
Authors       authors        text list   &    name   S  S  S  S   yes      a
Author Sort   author_sort    text list   &    name   -  H  -  S   yes
Comments      comments       text long               -  S  -  S   yes      c
Cover         cover          y/n >file        >jpg   -  S  -  S   yes
Date          date           date       yyyy-MM-dd   -  S  H  S   yes
Formats       formats        text list   ,           S  H  -  S   act      f
id            id             integer                 -  -  -  -    no
Identifiers   identifiers    text list   , label:x   H  H  -  S   yes
Languages     languages      text list   ,           S  H  H  S   yes
Marked        marked         yes/no      pin RowNo   -  -  -  -   act
Modified      last_modified  date       yyyy-MM-dd   -  H  H  -   act
On Device     ondevice       yes/no      checkmark   -  -  S  -   act
Path          path           text                    -  H  -  -   act
Published     pubdate        date          yyyy-MM   -  H  S  S   yes
Publisher     publisher      text                    S  S  H  S   yes
Rating        rating         integer         stars   H  H  H  S   yes
Series        series         text                    S  S  S  S   yes      s
Series Index  series_index   floatg pt     {0:.2f}   -  -  *  S   yes
Size          size           floatg pt     {0:.1f}   -  -  S  S   act
Tags          tags           text list   ,           H  H  H  S   yes
Title         title          text                    -  S  S  S   yes      t
Title Sort    title_sort     text                    S  H  -  S   yes
uuid          uuid           hex num                 -  H  -  -    no
  • Calibre Columns in Device View. These calculated fields are shown in Device View when a device is connected. See Metadata section for description of each column.
Code:
Calibre Columns                 S  Shown             Visibility
in Device View                  H  Hidden            ----------
                                -  not applicable    Tag browser   E
                                *  see description   Book details  d  Groupd
--------------------------------------------------   Library view  i  Search
                                         Separator   Edit dialog   t   Term/
Name          Lookup         Type        or Format   T  B  L  E    ?   Alias
-----------   ------------   ---------   ---------   ----------   ---   ----
Authors       authors        text list   &           -  S  S  -   act      -
Collections   collections    text list   ,           -  -  S  -    *       -
Date          timestamp      date       yyyy-MM-dd   -  S  S  -   act      -
In Library    inlibrary      yes/no      checkmark   -  -  S  -   act      -
Size          size           floatg pt     {0:.1f}   -  -  S  -   act      -
Title         title          text                    -  S  S  -   act      -
  • Custom Columns. Custom columns are user-defined columns that can be edited in Edit Metadata dialog, or hidden or shown in booklist or Book Details. When adding a custom column, the user enters the column label and column lookup name without the hash symbol prefix "#". Subsequently, in search criteria, the column lookup name includes the hash symbol prefix "#". These are the custom columns in my library L1. See Metadata section for description of each column.
Code:
Custom Columns                  S  Shown             Visibility
                                H  Hidden            ----------
                                -  not applicable    Tag browser   E
                                *  see description   Book details  d  Groupd
--------------------------------------------------   Library view  i  Search
                                         Separator   Edit dialog   t   Term/
Name          Lookup         Type        or Format   T  B  L  E    ?   Alias
-----------   ------------   ---------   ---------   ----------   ---   ----
aka           #aka           text list   &    name   S  H  S  S   yes      a
Awards        #awards        text list   ,           S  H  S  S   yes     aw
FormatQ       #formatq       text list   ,           S  S  S  S   yes     fq
Formats2      #formats2      composite           *  no  H  S  -    no      -
Genres        #genres        text list   ,           S  S  S  S   yes      g
Pages         #pages         integer         {0:,}   H  H  S  S   yes     pp
Types         #types         text list   ,           S  H  S  S   yes     ty
Read          #read          text list   ,           S  H  S  S   yes      r
Source        #source        text                    S  H  S  S   yes     so
  • Column-Specific Settings. Each column has associated column-specific settings. Some of them are:

    • Column Alignment. Alignment of metadata in the column left, center, or right is set per right-click menu command under column heading. My text, comma separated text, and date columns are left aligned. My number columns are center aligned for more pleasing visual spacing between adjacent columns.
    • Column Coloring. Column coloring rules color text in designated columns according to specific metadata values in a specified column, as set in Preferences > Look and Feel > Column Coloring. I use text colors to emphasize the status or quality of the format. See Metadata > Custom Columns > FormatQ.
    • Column Icons. Column icon rules assign icons to display in a designated column according to specific metadata values in a specified column. I do not use any column icons.
    • Column Size. The width of each column.
    • Column Visibility. Columns can be shown or hidden, per checkboxes in Preferences > Add Your Own Columns, or per right-click menu command under column heading. I hide the columns Date, Languages, Modified, Rating, and Tags. I hide Date because I adequately see it per book in Edit Metadata Individually and Book Details. I do not use Modified, Tags, and Rating because I prefer my own custom columns for similar purposes. I also do not use Languages. Occasionally I search a hidden column without showing the column. Note: I maximize horizontal space for seeing columns in the booklist by hiding the Tag Browser except temporarily while using it, and by setting Book Details below the booklist rather than to the right of the booklist.

Order of Columns
  • Presented Order. Library structure includes the presented order of columns.
  • Workflow Order. One practical way to order columns in the Library View booklist is by workflow steps for processing new books.
  • Workflow Clusters. In workflow-step order for processing new books, the columns cluster by general workflow task: Add books, Edit metadata, and Convert/Edit formats.
  • Level Groupings. It is also useful to group columns by the level at which the metadata elements are applied to books in bulk. I apply metadata at these levels:

    • Format-level. I keep only one format per book record because I want to avoid confusion arising from multiple formats and various format-level tags existing in one book-level database record. Using the rule "one format per book" allows assigning format-level metadata at book-level with confidence and without having to prefix each format-level metadata element with format type. I may have an additional format temporarily for a specific purpose, but I always know which format is the master digital format, I always remove the temporary format as soon as possible, and I know the format-level metadata elements apply to the master digital format, with one exception for the element indicating paper book owned.
    • Book-level. One authors/title combination can represent many separate publications, each a different publisher, language, edition, binding, media type, etc. Calibre uses a mix of title-level and publication-level. For example, duplicate matching occurs at authors/title level, but many of the Metadata Download fields are publication-level. So I combine publication-level metadata with title-level metadata into one level and call that book-level. When necessary for different publications with identical authors/title, I append publication-level metadata to the title to make authors/title unique, which equates to book-level unique.
    • Series-level. I incorporate series hierarchy with series name in just one Series column, and if relevant, also append a tag in bulk to series name indicating that the book is a member in a varied-author series. I also often apply genres at series-level, though I categorize genre as book-level. When entering metadata for a series by identical set of coauthors, I ensure the order of the coauthors in Authors is the same for every book within the series.
    • Author-level. In author-level custom columns I apply metadata pertaining to the first or only author listed in Authors, across all of that author's books.

  • Metadata Cliques. I prefer working with columns in one static all-purpose order. Switching between different column orders with View Manager plugin to achieve specific purposes seems dissonant to me. After some trial and error, eventually I embedded workflow-step order within metadata-level group within workflow-cluster then adjusted for practicality and look/feel. As a result, the columns congregated in stable cliques that are useful in a static-order all-purpose view of metadata. In my library L1, the metadata cliques are:

    • Core. Authors, Author Sort, Series, Series Index, Title, Title Sort, Date (date), Modified, id, uuid, Path, Marked.
    • Device. OnDevice, inLibrary [device], Authors [device], Title [device], Date (timestamp) [device], Collections [device], Size [device].
    • Fetch. Cover, Comments, Identifiers, Published, Publisher, Languages, Rating, Tags.
    • Book Custom. #Genres, #Types, #Awards, #Read.
    • Format. Formats, #Formats2, #FormatQ, #Pages, Size, #Source.
    • Author Custom. #aka.

  • Static Column Order in Library View. In Library View booklist the static order of my columns is: OnDevice; Authors, Series, Title, Date (hidden), Modified (hidden); Published, Publisher (hidden), Languages (hidden), Rating (hidden), Tags (hidden); #Genres, #Types, #Awards, #Read; #Formats2 (showing Formats), #FormatQ, #Pages, Size, #Source; #aka. The positions of #Read and #Source are special cases, each with its relevant clique but outside of workflow order. (See the Metadata section for details.)
  • Static Column Order in Device View. In Device View booklist the static order of the limited number of columns is somewhat congruent with the order in Library View: inLibrary [device], Authors [device], Title [device], Date (timestamp) [device], Collections [device], Size [device]. (See the Metadata section for details.)
  • Static Custom Column Order in Edit Metadata Dialog. I set the order of custom columns presented in the Edit Metadata dialog to be congruent with the Library View booklist order, per Preferences > Advanced > Tweaks > Order of custom columns in Edit Metadata.


Metadata
Spoiler:

Overview
  • Purpose. Describe the metadata scheme for my library L1, with specific column and metadata definitions and related usage conventions.
  • Topics. Calibre Columns in Library View, Calibre Columns in Device View, Custom Columns.


Calibre Columns in Library View (Definitions for Library L1): Authors, Author Sort, Comments, Cover, Date (date), Formats, id, Identifiers, Languages, Marked, Modified, OnDevice, Path, Published, Publisher, Rating, Series, Series Index, Size, Tags, Title, Title Sort, uuid.
  • Authors (authors). Calibre column, comma separated text, names, editable. For author name or a list of coauthor names. Rule: must contain at least one author name. I apply the following conventions for Authors:

    • Coauthors. For books with coauthors, ensure all of them are entered and in correct or desired order. The order of coauthors and the choice of which coauthor is first affects sorting the booklist by Authors in library, sorting on devices, and which author name is sent through metadata plugboards or save templates that use only the first author in Authors or Author Sort. For a series with books all written by the same set of coauthors, in Authors enter the identical order of coauthors for each book in that series. Calibre separates the names in the list with space ampersand space.
    • FirstName LastName versus LastName FirstName. I use the convention FirstName LastName. This is helpful with some internet sites. For example the International Speculative Fiction Database (ISFDB) site wants the FirstName LastName convention pasted into its Search box. One way to swap First and Last name is to use the plugin Quality Check, with search scope set for selected books, Fix menu, Swap author, while being mindful of the Tweak setting for Author sort name algorithm and the potential need to revise Author Sort in Manage Authors. See Author Sort below.
    • Author Initials. If necessary, I remove period (dot) after each initial or suffix in author name.
    • Author Last Name Prefixes. If there are prefixes in an Author last name, I correct any related Author Sort problems. See Author Sort.
    • Author Last Name Suffixes. If a last name has a suffix such as Jr, Sr, or III, in Authors delete the suffix along with any preceding comma or following period, then after downloading metadata enter the full name including suffix at the top of Comments. I delete the suffix in Authors because a comma before the suffix may confuse the Author Sort algorithm depending on how that is set, and when using an author template or directly copying/pasting to the search box of an internet site, the lack of the suffix also avoids some trouble with certain sites wanting or not wanting a comma before the suffix, the suffix itself, or the period. Removing the suffix from the name in Authors is a simple solution that does not work in all cases but avoids the need to implement more complex solutions for each site-specific author template, which still might not solve all cases for each site. See templates for various sites in the plugin Search the Internet.

  • Author Sort (author_sort). Calibre field, text, automatically filled, editable. Shown in Edit Metadata Individually. Calibre sets Author Sort automatically according to the Tweak Author sort name algorithm. I use author_sort_copy_method = 'comma' in conjunction with the FirstName LastName convention in Authors, and ensure there is no comma in Authors to interfere with the author sort. Rule: one to one correspondence with Authors. See calibre User Manual, FAQ, How does calibre manage author names and sorting?

    • LastName Prefixes. Potential sort problems can occur with prefixes that belong in the LastName such as: de, de la, del, le, van, St (regardless of upper or lower case). I want Author Sort to be set as the author likes it set, where possible, such as de la Cruz, Melissa or van Vogt, A. E. Consult a bibliographic site when unsure how to sort an author name (with the understanding that the site might be wrong).
    • Editing Author Sort. To fix Author Sort problems, select one book by any author, in Edit Metadata click the head and shoulders icon to the left of Author(s) field to get to Manage Authors, and manually correct Author Sort for any author with incorrect Author Sort.

  • Comments (comments). Calibre field, long text, editable in Edit Metadata Individually, shown in Book Details. For a short synopsis or review of the book, obtained through Metadata Download, or by copying/pasting from a bibliographic site. Links or additional notes can be added. For most books I prefer limiting Comments to one non-spoiler synopsis, with no reviews. Sometimes I add a link or a brief note at the top of Comments. Rule: a book can have zero to many synopses, notes, or links.
  • Cover. Calibre field, yes/no, in conjunction an image file. Calibre automatically stores the image file as cover.jpg along with the format in the book folder in the calibre library folder. The image file is editable and is displayed in Book Details, Edit Metadata Individually, Cover Grid, and Cover Browser. Obtained through Metadata Download, copy/pasted from a site (bibliographic, author, publisher), or by clicking the book icon in Edit Metadata Individually to set from the selected format. Calibre rule: zero to one cover per book. My rule: there must be a cover for each book, except for exclusion placeholders (see Metadata > Custom Columns > FormatQ).

    • Alternate Cover. When the correct cover for a specific publication is unavailable, use any available cover from any other publication of the book. If none are available use the standard calibre cover.
    • Editing Cover. Trim borders manually or automatically in Edit Metadata Individually.
    • Cover in Format. Depending on the type of format, calibre usually inserts the cover into the exported or conversion output format if necessary when calibre exports or converts a book. Insert the cover into the format in the library using Polish Books (EPUB or AZW3) or the plugin Modify ePub (EPUB). I generally insert cover and metadata into EPUB format with Polish Books near the end of the Processing workflow.

  • Date (date). Calibre column (in Library View), date, automatically filled, editable. Date the book was added to library. I format it as yyyy-MM-dd per the Tweak control how dates are displayed. I keep this column hidden, but show Date in Book Details. Calibre rule: must be one date in Date. My rule: edit only for good reason, while preserving the meaning of "date book was added."
  • Formats (formats). Calibre field, comma separated text, automatically filled, not editable, in conjunction with the format file. Calibre automatically stores the format in the book folder in the calibre library folder. Calibre displays the format names with filetype icons in the formats box in Edit Metadata Individually, where user can add or remove formats. Calibre rule: a book can have zero to many formats. My rule: after processing, a book can have zero to one format in Formats. For me this rule avoids confusion.

    • Original Format Tweaks. The tweaks in "save original file when converting/polishing from same format to same format" affect whether an original format is saved or not with Polish or same-to-same Convert. I set both to False for reasons explained Processing > Convert/Edit Formats.
    • Master Format. During processing, for EPUBs I fix any formatting problems and set styles to my preferred values, then remove all other formats for each book except the EPUB to keep as master format. In most cases (>99.99%) I keep only an EPUB. In rare cases (<0.01%) for currently useful reference material that has no good EPUB or AZW3 (to convert to EPUB) available at present, I keep only a PDF which I hope to replace in future. I do not convert PDFs, to avoid wasting time fixing conversion problems.

  • id (id). Calibre field, integer, automatically filled, not editable. The identification number of a book, unique within a library. Automatically assigned in increments of 1 when book is added or copied to a library. In the library folder calibre also appends "id" onto title folder. In the save template for Saving Books to Disk, I append "id" onto the title folder to ensure there will be no author/title duplicates going to the file system. Rule: one library-unique id per book.
  • Identifiers (identifiers). Calibre field, comma separated text, automatically filled by Metadata Download, editable in Edit Metadata Individually, in the form label:number. For ISBN10 or ISBN13 and other identifiers such as retailer-id for that book. Calibre constructs identifiers into links to the respective page for that book on the relevant site, clickable in the Book Details pane. Before Metadata Download, extracting a good ISBN from the format with the plugin Extract ISBN can help get metadata specific to the edition with that ISBN. Otherwise, delete identifiers first before doing another Metadata Download to try for better metadata. Generally I do not edit Identifiers, do not care if there is an ISBN or not, do not care if an ISBN is ISBN10 or ISBN13, and seldom use the links in Book Details. (For books where I have multiple editions, for each I append relevant edition-identifying metadata in parentheses in Title, for example (tr Clavell) or (3rd ed), to make Title unique.) Rule: there can be zero to many identifiers per book.
  • Languages (languages). Calibre column, comma separated text, editable. For the language of the book. I fill this column through metadata download, and hide the column because I do not use it. Calibre rule: zero to many languages per book.
  • Marked (marked). Calibre field, yes/no, not directly editable. Shown with a pushpin next to the row number for each marked book. Set or cleared through the Mark Books feature, or through several optional plugins such as Find Duplicates. I do not use this field except indirectly with those plugins. Marks set by Find Duplicates are the only marks that persist through a calibre restart; otherwise marks are for temporary use. Rule: zero or one pushpin per book.
  • Modified (last_modified). Calibre column, date, automatically filled, not editable. Shows the last time changes were made to metadata or cover, or formats were added/removed. I format Modified as yyyy-MM-dd per the Tweak control how dates are displayed. I sort dates by visible fields, per the tweak Choose whether dates are sorted by visible fields. I hide this column. Rule: each book has one last modified date, starting the same as Date when first added to the library then subject to subsequent changes.
  • On Device (ondevice). Calibre column (in Library View), text, automatically filled, not editable. Column appears when device is connected. Shows checkmark for books in the library which are also on the connected device, along with location on device (main or memory card). Changes by adding or removing books from device. Rule: one checkmark and location for each book in library which is also on currently connected device.
  • Path (path). Calibre field, text, automatically filled, not editable. The path to the book in the book folder in the library folder. Open the book folder with keyboard shortcut O. Alternatively, if Path is shown in Book Details, open the book folder with click to open. Path changes automatically with edits to Titles or Authors, and with move of library followed by pointing calibre to the new location. Rule: each book automatically has one unique Path.
  • Published (pubdate). Calibre column, date, editable. For date of publication. Obtained through Metadata Download, the publication date may need to be corrected per the publication data on a bibliographic site. I use date of first publication for fiction or popular nonfiction, or the date of a specific revised edition when that is important such as for technical books. I format the column as yyyy-MM per the tweak control how dates are displayed. I sort dates by visible fields, per the tweak Choose whether dates are sorted by visible fields. Calibre rule: zero to one publication date. My rule: must be one publication date for each book except exclusion placeholder.
  • Publisher (publisher). Calibre column, text, editable. For name of publisher of the book. Obtained through Metadata Download. Calibre rule: zero to one publisher per book. My rule: must be one publisher for each book except exclusion placeholder.
  • Rating (rating). Calibre column, integer, shown as stars, editable. The scale goes from zero to five stars, five the highest rating. I do not use this column, and keep it hidden. Calibre rule: must be one rating per book.
  • Series (series). Calibre column, text column for keeping series-like information, editable. The column displays series name (text), followed by bracketed series index (floating point number), both editable. Calibre rule: zero to one series name and index per book. My rule: must be one series name and index per book. The conventions I use in Series are:

    • Standardized Series Name. Enter or correct Series name and index according to the series name and index used by a bibliographic site with consistent series metadata. This is important particularly for series with books by varied authors, for which the series name and indexing should be consistent across various authors.
    • Individual Books. For individual books that are not in a series, I enter (1) in Series, and 1 for Series Index. This displays as "(1) [1]" in the Series column, sorts above other series names in the booklist, and displays as "Book 1 of (1)" in Book Details. A book has a blank series only immediately after Add, until series is entered. My rule: there must be at least one series name for each book, even if it is an individual title not actually in a series.
    • Hierarchical Series. When necessary, I use this convention for large series with multiple levels of sub-series: unique Broadest Series Name Abbreviation; sub-series name; most specific Sub-sub-series name. I prefer the appearance of semicolon space over the standard calibre convention of period no_space preceding each child subgroup. (See calibre User Manual, Managing subgroups of books.) For me the period preceding a subgroup is not necessary because I do not need to see a Tag Browser breakdown of series by subgroup. I apply the series index for the most specific sub-series exactly as given at the ISFDB or Fantastic Fiction site, with another higher level of indexing accomplished when necessary by appending to a higher-level series name a number that sorts sub-series in the desired order. I search for the Broadest Series Abbreviation or a relatively unique word or phrase contained in Series. Generally I avoid large series with multiple levels of sub-series.
    • Unique Series Name. Ensure there are no duplicate series names, except for author-level exclusions or individual books with series name (1) and index 1. For duplicate series names other than (1), ensure each is unique by appending parentheses containing author last name or initials.
    • Varied Author Series. When a series contains books with different author(s) between at least two of the books, for all books in that series append to series name a space and dollar sign " $". This provides a clue to see the entire series with sort by title within series, rather than sort by title within series within author. Generally I avoid varied-author series with books by more than three different authors or sets of coauthors.

  • Series Index (series_index). Calibre field, floating point number, formatted for 2 decimal places {0:.2f}, editable. Displayed in conjunction with series name in Series column, and separately in Edit Metadata Individually. Rule: if there is a series name, there must be a series index number, ranging from 0 to higher (no negative numbers). I use these conventions for series index:

    • Publication or Internal Order. Many series have books where the publication order of the books is the same as the internal order of events for the characters. If those orders are different, I may change the series names and series index from publication order to internal order, if it does not have too many hierarchical levels, I have all of the books to date, and the recommended internal chronology is published on an official site or in one of the books. For example, for Valdemar series by Mercedes Lackey, I changed all the small series names to Valdemar and indexed the books with relevant Valdemar calendar year; for the Recluse series by L. E. Modesitt Jr, I indexed using the relevant Recluse calendar year.
    • Auto-Increment. I leave at default the tweaks for auto-increment series_index: series index auto-increment = 'next', and use series auto-increment tweak when importing = False.
    • Indexing per Publication Type. I apply different indexing conventions depending on type of publication. For details, see Metadata > Custom Columns > Types.
    • Exclusions. For exclusion placeholders (empty books) at series or author level, I use index 0. I use series name for series level exclusions, or (0) as the series name for author level exclusions. For details, see Metadata > Custom Columns > FormatQ.

  • Size (size). Calibre column (in Library View), floating point number, formatted for one decimal place {0:.1f}, automatically filled, not editable. The size in megabytes of the largest format in the book folder, automatically revised when formats are added or removed using calibre commands. If any editing is done outside of calibre to that format while it resides in the calibre library folder, the value shown in Size column may not show the changed size until after running the command Fix > Check and repair book sizes with the plugin Quality Check. Size is not an accurate indicator of book length, because size may include images (including large-size high-resolution image files), embedded fonts, and other potentially size-varying components of the format.
  • Tags (tags). Calibre column, comma separated text, editable. For tags. I hide this column. I do not download tags in Metadata Download, because assigning my own tags in specific custom columns saves time by not having to correct or delete downloaded tags, while allowing better consistency and organization. Calibre rule: a book can have zero to many tags in the Tags column. My rule: a book must have zero tags in the Tags column, except for at most one calibre-automatic tag. In my usage, the only metadata elements in Tags are:

    • News. News download, automatically tagged by calibre. This is technically a publication type, but since calibre already tagged it here, I do not take the time to duplicate the tag in custom column Types. When necessary I search for tags:news.
    • Catalog. Catalog of books in book format, automatically tagged by calibre. This is technically a publication type, but since calibre already tagged it here, I do not take the time to duplicate the tag in Types. When necessary I search for tags:catalog.

  • Title (title). Calibre column, text, editable. Calibre uses Title as one component of a book format's filename when storing it in the operating system's file system. Calibre truncates the title in filename and changes non-compliant characters to underscore as necessary to stay within system limitations, while leaving Title unchanged in the database as seen in the booklist or Book Details. For information about system limitations for path and filename, see Links > Filename Limitations. Calibre rule: each book must have one title. My rule: the combination of Authors plus Title must be unique at book-level within the library. I use these conventions for Title:

    • Subtitle. I delete subtitle, unless the bibliographic site uses the subtitle as part of the title, in which case I change the preceding (colon)(space), to (dash)(space). Note: calibre automatically changes colon to underscore in the filename.
    • Unique Authors/Title. After Metadata Download, if necessary I append brief publication metadata to the title in parentheses to make the authors/title combination unique, in cases where alternate editions, versions, or titles are available now or will be in future, for example: (3rd Ed), (Anniv Ed), or (vt The Variant Title). I avoid a lengthy title by entering any lengthy information in Comments. Other users may employ custom columns for Edition or Title2, or use separate empty book records for alternate titles.

  • Title Sort (title_sort). Calibre field, text, automatically filled, editable. Shown in Edit Metadata Individually. I use the sort-order tweaks 'strictly_alphabetic' to control sorting of titles and series in the library display, and control formatting of title and series when used in templates. Rule: one to one correspondence with Title.
  • uuid (uuid). Calibre field, hex number, automatically filled, not editable. Can be seen in Book Details. uuid is a hex identifier, practically unique in the universe. Calibre uses uuid internally. Rule: one uuid per book.


Calibre Columns in Device View (Definitions for Library L1): Authors [device], Collections [device], Date (timestamp) [device], InLibrary [device], Size [device], Title [device] — all calculated metadata determined by internal calibre matching and decision making when device is connected.
  • Authors (authors) [device view]. Calibre column, comma separated text, names, filled automatically, not editable. Shows authors for books on device. Rule: one or more authors per book.
  • Collections (collections) [device view]. Calibre column, comma separated text, potentially automatically filled, potentially editable. Set up with metadata management options in Preferences > Sending books to devices, in combination with any relevant tweaks, perhaps with the aid of a specific device interface plugin, depending on specific device. Collections are editable in Device View for some combinations of those options for some devices. I do not use collections on devices at all because: first, I load devices only a few books at a time so do not need collections to organize them there; second, collections cannot be used in a standard way across all my devices because they are device-dependent, requiring in some cases various complicated setups including a dedicated custom column, and collections support from calibre is not available for all devices. Rule: zero to many collections per book.
  • Date (timestamp) [device view]. Calibre column, date, automatically filled, not editable. The date the book was sent to device from calibre or otherwise arrived on the device. I format it as yyyy-MM-dd per the Tweak control how dates are displayed. Rule: one date per book.
  • In Library (inlibrary) [device view]. Calibre column, yes/no, automatically filled, not editable. Shows with checkmark which books on the device are also in the library. In debug mode, also shows method used in matching. Rule: zero or one checkmark per book that is also in the currently open calibre library while the device is connected.
  • Size (size) [device view]. Calibre column, floating point number, formatted for one decimal place {0:.1f}, automatically filled, not editable. The size in megabytes of the format on the device. Rule: one size per book.
  • Title (title) [device view]. Calibre column, text, filled automatically, not editable. Shows title of book on device. Rule: one title per book.


Custom Columns (Definitions for Library L1): aka, Awards, Formats2, FormatQ, Genres, Pages, Read, Source, Types.
  • aka (#aka). Custom column, comma separated text, editable, contains names. For alternate name(s) of the first or only author in Authors, separated by space-ampersand-space, using the same conventions as in Authors. The acronym aka means "also known as". Rule: a book can have zero to many aka names.
  • Awards (#awards). Custom column, comma separated text, editable. For winning award in the novel category. I do not track awards for categories other than novel. Once a year, I review the awards for best novel on Wikipedia, for awards given the previous year (and prior years if necessary). If any of the winning novels or novel runners-up are not yet in the library, I buy and process them, then enter the award name for all owned books that won the award for novel. Updating the library for awards is an annual Maintenance task. Rule: a book can have zero to many awards. The metadata elements in Awards are:

    • b, Bram Stoker award for best novel. Awarded 1987-present. March-June.
    • c, Campbell Award for best novel. Awarded 1973-present. June.
    • h, Hugo Award for best novel. Awarded 1953-present, except 1954 and 1957. September.
    • if, International Fantasy Award for best novel. Awarded 1951-1955 and 1957 only. Defunct.
    • n, Nebula Award for best novel. Awarded 1966-present. May.
    • pz, Pulitzer Prize for fiction (previously for novel). Awarded 1918-present. April.
    • wf, World Fantasy Award for best novel. Awarded 1975-present. October.

  • FormatQ (#formatq). Custom column, comma separated text, editable. For Format Quality rating and any other format-quality or format-status tags. Based on assessment of format after adding book and fixing format problems or adjusting styles, the format quality rating enables format quality control, provides a basis for comparison between duplicates, identifies books with no formats or annoying formats that can be replaced in the future, and ultimately applies only to one master digital book format. The rating scale is based on types of empty book (with no format to read) for the bottom ratings, then moving up the scale through worst to least amount of negative emotion caused by formatting issues when reading a format. Rule: a book can have one to many format-related metadata elements, and one of them must be format quality rating. The metadata elements are:

    • f0, Exclusion placeholder, do not acquire format ever, empty book (no format). Useful to prevent wasting time reinvestigating in future. I set Column Coloring to color text tan in all columns of each exclusion placeholder to visually distinguish the record in booklist. I set Create Catalog to indicate exclusion with "x". For each new exclusion placeholder, enter "f0" in FormatQ and make the Title unique by constructing it as follows: "_xLevel; reason_excluded; level_element", where Level is T, S, or A for Title, Series, or Author; reason_excluded is a very brief reason; and level_element is the title, series, or author. For title exclusion enter Authors and Series as usual. For series exclusion enter Authors and Series as usual but 0 for Series Index. For Author exclusion enter only the excluded author in Authors, (0) in Series, 0 in Series Index. Remove any format(s) in the book record of each exclusion placeholder. Example Title for each level of exclusion:

      • Title-Level. _xT; bad writing; the book's title
      • Series-Level. _xS; poor & boring; name of series
      • Author-Level. _xA; ro (romance); name of first author in Authors

    • f1, Wishlist placeholder, acquire format in future, empty book (no format). When adding new wishlist placeholders, enter Authors, Series, and Title as usual then process the empty books as if they were real books for everything except format-related tasks. When a format is finally acquired, drop the format into this wishlist record then process it like any other newly added book, revising previously entered metadata where relevant. I set Column Coloring to make text magenta in all columns of each wishlist placeholder to visually distinguish the record in booklist. I set Create Catalog to indicate it with "w".
    • f2, Not readable, or is readable with strong annoyance, anger, rage, or despair due to formatting problems impossible to fix, or not worth the time to fix. Immediately remove format, and enter "f1" indicating wishlist placeholder. Never assign "f2".
    • f3, Readable with mild annoyance. Some mildly annoying formatting problems remain because they are not possible to fix quickly at current knowledge/skill level. Or the format is a pre-release advance or review copy. The rating "f3" means the format is acceptable for now but replace it with better in future. I set Column Coloring to make text mediumvioletred in all columns of an "f3" book record to visually distinguish the record in booklist.
    • f4, Readable with no annoyance. Assign after assessment, adjusting styles to personal preference, and fixing all formatting problems. Most formats in my library are "f4" (approx 80%, the other 20% are f3).
    • f5, Readable with awe. Excellence of formatting is a moving target, due to gradually increasing knowledge of formatting and skill adjusting styles and fixing formatting problems. Problems announce themselves loudly, as compared to quiet and unobtrusive excellence of formatting without perceived problems, possibly not consciously noticed at all. So for practicality I assign "f4" instead of "f5".
    • h1, Format problem: the CSS does not distinguish between chapter headings (or section headings) and the following text. Unable to automatically fix in the CSS with search/replace, and too time consuming to manually fix. Rate format quality as f3 or worse, depending on annoyance level.
    • pb, Own the paper book. This metadata element is independent of format quality rating or other format-related elements. It is also the only element in FormatQ that does not pertain to an ebook format.
    • rc, Review copy, advance reader copy. I want to replace this format with the proofed final-release version when available, so I also assign a format quality rating of f3 even if there are no formatting problems.
    • sb, Format problem: missing section or scene breaks. Unable to figure out where the breaks should be with the information in this format, or whether breaks are supposed to be there or not. For example, no asterisks, no single-empty-line, and no double-empty-line exist to separate sections of text within any of the chapters. This is tricky due to the plethora of ways to indicate scene breaks, and the possibility that no scene breaks were wanted by the author or editor. Initially rate format quality as f3, and after reading the book downgrade or upgrade the rating as relevant.
    • spl, Split. This book format was split (with optional plugin EpubSplit) from a larger book with multiple components such as an omnibus. In Source, use the source of the original book that was split.

  • Formats2 (#formats2). Custom column; built from other columns, behaves like tags; template {:'approximate_formats()'}, sort/search column by text, unchecked show in tag browser, unchecked show as HTML in book details. Automatically filled, not editable. For seeing formats at a glance in the booklist. Named Formats2 to avoid potential confusion with the calibre field Formats. During library startup, the template {:'approximate_formats()'} populates the field from format metadata in the database, which is fast but would be inaccurate if any formats were added or deleted directly in the library folder in an operation unknown to calibre (which I avoid doing). In comparison, the template {formats} looks at the format files in each book folder of the library in the filesystem, which is potentially more accurate but requires more time during library startup.
  • Genres (#genres). Custom column, comma separated text, editable. For high level genre classifications. I do not download tags. Instead I determine genres by context with other books in a series; by assessing key words and phrases in Title, Series, and Comments in the book record, by sometimes assessing genre tag or synopsis on the title page of a bibliographic site, or if all else fails, by reading a few pages of the book. Rule: a book must have at least one, and no more than two genres; preferably just one. This genre classification is solely for my own convenience. The metadata elements are:

    • adv, Adventure. Action in exotic places, usually not contemporary.
    • fn, Fantasy. All sub-genres except urban fantasy.
    • hor, Horror. Focused on instigating horror or terror.
    • main, Mainstream. Not marketed as a specific genre.
    • mys, Mystery. A sleuth investigates to solve a crime; other crime fiction is thriller.
    • par, Paranormal romance. Supernatural characters in historical or contemporary setting, and emphasis on romantic relationship.
    • ro, Romance. All sub-genres of romance except paranormal.
    • sf, Science fiction. All sub-genres.
    • thr, Thriller. Suspense, contemporary action, excluding horror.
    • uf, Urban fantasy. Usually contemporary, urban, with characters with supernatural or otherwise fantastic abilities, and emphasis on issues other than romantic relationship.
    • ws, Western. Action in the old wild west of North America, usually sometime between 1850 and 1890.
    • nf, Nonfiction. Based on real people, events, and facts. Note that International Speculative Fiction Database treats nonfiction as a publication type (Help:Screen:NewPub, ISFDB wiki, definitions for publication-level metadata.)
  • Pages (#pages). Custom column, integer, formatted for comma {0:,}, editable. For the number of pages in the book, computed by the plugin Count Pages, using algorithm Paragraphs (APNX accurate). I do not use the plugin's word count or readability capabilities. I do not care whether the page count on device is the same or not. Page count is a good indicator of book length, while file-size in the Size (MB) column is not. For example, page count helps determine whether a book is a novel or a separately published short with the same author and title; the short may have a high resolution cover making the short's file-size greater than the novel file-size. Rule: each book record with a format must have a page count.
  • Read (#read). Custom column, comma separated text, editable. For content rating entered after reading each book, and for "to read" tag to identify books to read or reread soon. The presence of a content rating means I read the book. No content rating means either I have not read the book, or I read it too long ago to enter a meaningful content rating. For books read more than once, the content rating represents the last time the book was read. Rule: a book can have zero to one "to read" element and zero to one content rating, with maximum two metadata elements. The metadata elements for Read are:

    • TR, To Read. (This is included in the saved search "To Load".)
    • r1, Content rating: bad. The writing is bad.
    • r2, Content rating: poor. The writing is poor.
    • r3, Content rating: okay. The writing is okay. Some boring parts.
    • r4, Content rating: good. The writing is good. Interesting. Not boring. Read it all.
    • r5, Content rating: excellent. The writing is excellent. Fascinating, gripping. Read it all.

  • Source (#source). Custom column, text, editable. For the name of download site, retailer, or other source of the format. In workflow Source is entered immediately after Add Book, rather than later when processing the other format-level fields. Rule: each book with a format must have one source. Some (of many) metadata elements in Source are:

    • Amazon, Amazon US site.
    • AmzKU, Amazon Kindle Unlimited.
    • Author, Author site or self-published (refers to name in Authors).
    • Baen.CD, Baen CD at the site Fifth Imperium. 23 free CDs each with multiple books.
    • Baen.Eb, Baen Ebooks at baenebooks.com.
    • Baen.FL, Baen Free Library at the Baen Ebooks site.
    • B&N, Barnes and Noble.
    • BookView, Book View Cafe.
    • MR, MobileRead. Free public domain books with good formatting.
    • O'Reilly, O'Reilly Media. Technical books.
    • ProjGute, Project Gutenberg US site. Free public domain books.

  • Types (#types). Custom column, comma separated text, editable. For publication types. All publication types except novel are rare in my library, because I prefer reading whole text-based novels, rarely add or keep the other types, and make exceptions for only a few authors or publications. Rule: zero to many per book. The metadata elements in Types are:

    • an, Anthology. An anthology contains shorts by various authors. For anthologies with credited editor(s), in Authors I do not include the authors of content but use the names of the editor(s), without appending (ed) to the names. For anthologies without any credited editor I list the authors of internal content, for example for teaser anthologies of paranormal romance novellas each with different authors. For Series Index, treat anthologies as if they were novels, or index them as 301, 302, 303, depending on whether the timeframes of the contained shorts easily fit into novel indexing or not.
    • col, Collection. A collection is a group of shorts all by one author, or all by one set of co-authors. For Series Index, treat collections as if they were novels, or index them as 201, 202, 203, depending on whether the timeframes of the contained shorts easily fit into novel indexing or not.
    • cmc, Comic book. Image-based comic book or graphic novel. For Series Index, number graphic novels or comics like novels, or use issue numbers.
    • mgz, Magazine. Periodical, usually a combination of image-based and text-based content, often a combination of fiction and nonfiction. In Authors enter abbreviation of magazine name. In Title enter magazine name and issue number. In Published enter issue date.
    • nvl, Novel. Fictional events and characters in a long prose-based narrative, with length 40,000 words or more (the standard to qualify for the category of novel in the Hugo or Nebula awards). Absence of a Types tag indicates the type is novel, or the type is tagged elsewhere (news or catalog in Tags, nonfiction in Genres). Most of my books are novels, so for convenience I enter the novel tag only when I have duplicate authors/title books each with a different type, for example a short story and a novel, in which case I also append to Title the relevant type tag in parentheses, (shr) or (nvl), to make each Authors/Title unique at book-level. For Series Index, use whole numbers, beginning with 1 for the first novel in the main series; for prequels use 0 with decimals: 0.01, 0.02, 0.03.
    • nvz, Novelization. A novel based on a previously existing video game or movie.
    • om, Omnibus. A compilation containing at least one novel, usually two or three, often accompanied by one or more shorts. The contents may be by one or several different authors, in the latter case usually in the same series. I replace an omnibus with the separate novels, or when those are not available I split the omnibus into separate novels with the plugin ePubSplit, preserving the copyright information for each novel, then remove the omnibus. Separating the novels simplifies series numbering and author updating. If for some reason an omnibus is kept rather than split, in Series Index use numbers higher than the novel numbers such as 123, 456, 789; or alternatively 101, 102, 103 for a series with more than 9 omnibus-contained novels.
    • shr, Short. I use the term "short" for short story, novella, novelette, chapbook, or essay. This is purely for my convenience, because I prefer novels and rarely want any of the "short" types. In Series Index, use decimal number indicating placement between relevant novels in the series, for example 3.51 and 3.52 for two shorts belonging between novels 3 and 4.
    • srl, Serial. A serial is published in parts, usually in consecutive issues of a magazine. The combination of parts is usually novel-length. I do not collect them unless a separately published novel is not available and it is likely that I can gather all the serial parts. After collecting the parts I use plugin ePubMerge to merge the parts into one book. In Series Index, indicate parts of a serial with decimals after the relevant novel number, for example three serial parts of novel 6 would be 6.01, 6.02, 6.03.
    • Catalog, Catalog in book format, listing 2 or more books in a library. Calibre automatically assigns this metadata element in Tags, so I do not enter it in Types.
    • News, News download. Calibre automatically assigns this metadata element in Tags, so I do not enter it in Types.



Links
Spoiler:

Overview
  • Purpose. The Links section provides links to ebook-related information and tools.
  • Topics. Calibre (various subtopics), Bibliographic Sites, Catalog on Server, Conversion, CSS, Devices, DRM, eBooks, Filename Limitations, Filename Renamers, Format Editing, Formats (subtopics per format), HTML/XHTML, Icons, Images, Metadata Terms, Regular Expressions, Searching, Strategy, Templates, Version History.
  • Links Key. The links are color-coded.

    • Green. Information on MobileRead Forums.
    • Purple. Information on other websites.
    • Blue. Software tools that may be useful with ebooks. Tools recommended by experienced people at MobileRead Forums are tagged recommended. Any I have not tried myself are tagged noted.

Calibre: Download
Calibre: Getting Started
Calibre: MobileRead Forums
Calibre: User Manual
Bibliographic Sites
Catalog on Server
Conversion
CSS (Cascading Style Sheets)
Devices (Reading Devices)
DRM (Digital Rights Management)
eBooks
Filename Limitations
Filename Renamers
Format Editing
Formats: EPUB (Electronic Publication)
Formats: Kindle* (AZW3, MOBI, . . .)
Formats: PDF (Portable Document Format)
Formats: Word Processor — TXT, RTF, DOC, DOCX, ODT (Text, Rich Text Format, Document, Document XML-based, Open Document Text)
HTML/XHTML (HyperText Markup Language / eXtensible HyperText Markup Language)
Icons
Images
Metadata Terms
Regular Expressions (regexp, regex):
Searching
Strategy
Templates
Version History

Last edited by unboggling; 07-30-2015 at 01:21 PM. Reason: Ongoing revision.
unboggling is offline   Reply With Quote
Old 07-19-2012, 04:15 PM   #2
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Thread Rules

Please post elsewhere any questions about how calibre works, or any requests for help with a problem, by starting a new thread in the main calibre forum or relevant calibre subforum.

Please post in this thread feedback about the first post, recommended changes to it, or comments about different ways to manage ebooks with calibre.

Last edited by unboggling; 02-15-2014 at 09:55 AM. Reason: clarify.
unboggling is offline   Reply With Quote
 
Advertisement
Old 07-19-2012, 04:45 PM   #3
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 8,455
Karma: 51865024
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
@Unboggling; any thought into putting this into an alternate format, maybe (oh horrors) an ePub?
PeterT is offline   Reply With Quote
Old 07-19-2012, 05:00 PM   #4
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Quote:
Originally Posted by PeterT View Post
@Unboggling; any thought into putting this into an alternate format, maybe (oh horrors) an ePub?
I'd thought about it. Generating an ePub would be relatively easy to do. But it seems like it's more accessible online here than in an ePub that might be added to someone's library then forgotten. At least if I added it to my library, I'd probably forget it was there. It's easier for me to deal with frequent revisions by just pasting new version over older version at Post #1 of this thread.

Last edited by unboggling; 07-20-2012 at 01:05 PM. Reason: Minor clarification
unboggling is offline   Reply With Quote
Old 07-21-2012, 09:12 PM   #5
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Quote:
Originally Posted by PeterT View Post
...maybe (oh horrors) an ePub?
Giving a book version further thought:

Translating the BB code formatting of the version at Post #1 into HTML is relatively painless, per suggestion of theducks to copy the HTML of the post page and paste it from there into an editor, but then spoilers need to be stripped and formatting redone to be more suitable for a book. I need to revise this document frequently with changes to fix unwitting errors and keep it congruent with my actual settings, customizations, and workflow. A book version doesn't seem practical.

Last edited by unboggling; 07-22-2012 at 12:25 AM.
unboggling is offline   Reply With Quote
Old 08-13-2012, 05:35 PM   #6
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Direction of "How I" Project

The content of Post #1 now seems relatively stable and problem-free, though it will still need revisions for changes in calibre or in my workflow. Please let me know if you find any problems I missed, so I can fix them. Meanwhile I've been thinking about the future direction of this project.

1. Searching the forums works best for small posts, when each post is limited to one specific topic. This large document has lots of topics, so it pops up often when using forum-search for keywords, which may annoy people searching for solutions to specific problems. That brings up the post but doesn't show where keywords are in the document unless the spoilers are expanded to allow looking for the highlighted keywords. Alternatively, if all spoilers are expanded first, a browser search of the document for keywords works well. Without expanding spoilers first, a browser search for keywords when viewing the page source also works but shows any keyword hits from HTML tags. All the search methods are awkward for one reason or another.

2. If I transform this project into a website:
  • + People could link to it from a post or sticky here, avoiding the search problem here.
  • + Site would have internal search feature.
  • + HTML and CSS would be more flexible than BBcode for formatting.
  • + After each revision of content, it could be transformed into an ebook with significantly less work than from the BBcode-formatted document here.
  • - Implementation would require learning more HTML and CSS along with the initial work of setting up the site. Good for me in the long run for eventual use with ebooks, but still time-consuming.
3. I'm not sure how useful this project is to other people. The content is different from and broader than the usual calibre forum content of "how do I do this new-feature-to-me?" or "how do I solve this small specific problem?" Last year, as a new user I looked for content like this, overall maps of how other people used calibre, including strategy and workflow along with examples. Couldn't find anything as broad as I wanted, just disorganized bits and pieces throughout these forums. So started the KISS thread, and that eventually grew into this document. I still wonder if others find it useful, if there's a common need for this kind of content.

What do you think? Feedback will help me decide what to do. Please comment, either by private message or in this thread.

Last edited by unboggling; 08-17-2012 at 12:36 PM. Reason: details about searching
unboggling is offline   Reply With Quote
Old 08-30-2012, 03:34 PM   #7
brainvision
Enthusiast
brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.brainvision ought to be getting tired of karma fortunes by now.
 
brainvision's Avatar
 
Posts: 47
Karma: 603120
Join Date: Aug 2012
Location: Monte Los Angeles (Califoggia)
Device: Android Smartphone
I already gave you karma points - as you know - 'cause I found this thread very very useful: I think that it contains a lot of information that everybody needs! Once starting using Calibre more in depth, than maybe will not follow al of your suggestion, but sure they are excellent example to show us what Calibre can do and how we should think at this program.

I prefer to read the post little by little, a paragraph at once,to be sure to really understand it and with the intention to try by myself your approach..
I don't know what you should do to give more emphasis to this post, but I'm pretty sure you should do it! And, more, if you can and if you want, you should try to make it always better and complete!!
Examples are always the best way to learn when we are talking about software and practical things to do..

thank you very very much, once again!
I really appreciate this "little" but precious work!
see you!!
brainvision is offline   Reply With Quote
Old 08-30-2012, 04:23 PM   #8
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
You're welcome. Glad to hear you find it useful. Thanks for the glowing endorsement. Made my day.

Edit:
And thank you again for the karma last week. I like karma. And I like feedback even more. I plan to continue revising and adding to it, at least for awhile. I'm still thinking about the "give more emphasis to this post" thing, as well as future direction of the project.

Last edited by unboggling; 08-31-2012 at 02:27 AM. Reason: added comments.
unboggling is offline   Reply With Quote
Old 09-01-2012, 11:07 PM   #9
travger
Evangelist
travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.
 
travger's Avatar
 
Posts: 469
Karma: 270594
Join Date: Aug 2010
Device: palm tx, Windows XP, Windows7
OK, I have some thoughts:

'Assessing Formats' - Bulk Convert to Preferred Format.
Seems a bit pointless to me. Pros - makes nice uniform library.
Cons - 1. Book that I fix today and feel happy about coloring headings purple may seem rather ghastly few years hence.
2. Any conversion may lose or add something (I'm presently reading a mobi book where there's no blank lines for the scene breakes, I doubt that they will appear after conversion. This particular book has such a twisted css that I must uglify very nice ePub in order to replace some code especially for the conversion to mobi. In this case I'll keep both nice formats).
3. When I don't read the book immediately, leaving original format be saves lot of time - just quick quality check is all.
4. By the time I get around to reading it, several things may have happened: I may have found better quality/preferred format; I may have gotten new reading device; my html skills will definitely increase.
5. Maybe next book in the series has different but nicer formatting, then I can change first one more easily, having an example.

Fixing Formats - Convert to Editable Format
IMHO converting anything html to word and back is not good practice. Editing for example ePub in Sigil is much more under control, it won't accept mistakes, and you can see right away the change (and maybe undo it).

Editing Metadata - Title
I prefer shorter title. Probably it's subconcious answer to Windows path length limit. A friend gave me his zipped library, and many books there were unextractable because the title was too long.

Maintenance -
Maybe warning about not to mess around directly in the Calibre folders should be capitalized?
travger is offline   Reply With Quote
Old 09-02-2012, 05:02 AM   #10
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Quote:
Originally Posted by travger View Post
OK, I have some thoughts:

'Assessing Formats' - Bulk Convert to Preferred Format.
Seems a bit pointless to me. Pros - makes nice uniform library.
Cons - 1. Book that I fix today and feel happy about coloring headings purple may seem rather ghastly few years hence.
2. Any conversion may lose or add something (I'm presently reading a mobi book where there's no blank lines for the scene breakes, I doubt that they will appear after conversion. This particular book has such a twisted css that I must uglify very nice ePub in order to replace some code especially for the conversion to mobi. In this case I'll keep both nice formats).
3. When I don't read the book immediately, leaving original format be saves lot of time - just quick quality check is all.
4. By the time I get around to reading it, several things may have happened: I may have found better quality/preferred format; I may have gotten new reading device; my html skills will definitely increase.
5. Maybe next book in the series has different but nicer formatting, then I can change first one more easily, having an example.

Fixing Formats - Convert to Editable Format
IMHO converting anything html to word and back is not good practice. Editing for example ePub in Sigil is much more under control, it won't accept mistakes, and you can see right away the change (and maybe undo it).

Editing Metadata - Title
I prefer shorter title. Probably it's subconcious answer to Windows path length limit. A friend gave me his zipped library, and many books there were unextractable because the title was too long.

Maintenance -
Maybe warning about not to mess around directly in the Calibre folders should be capitalized?
@travger. Thank you for the feedback. It helps me. A lot.

Assessing Formats - Bulk Convert.
I will clarify what options I use in conversion. The issues of bulk convert and deleting original format both need careful reconsideration. I'll at least add some discussion of pros/cons for each of those issues, both of which I've seen discussed here more than a few times. I vaguely recall we also talked about some of this last year, and revisiting these issues now from the perspective of a year's additional experience will be good.

Fixing Formats - Convert to Editable Format
I agree, that needs some clarification and rethinking too. I've been learning HTML, CSS, and Sigil gradually and am probably to the point where I should ditch fixing in Word, OpenOffice, or LibreOffice in favor of preferably fixing in Sigil in my real-life workflow. In the workflow document I was thinking of changing the simple fix example from Word back to OpenOffice, but first I need time to figure out and test how to anchor graphics to paragraphs in OpenOffice. I wanted to use a simple example. And I wish I'd started learning ins and outs of Sigil sooner. I'll rewrite this section into at least 2 fix subsections, one for Sigil fix, one for OO fix.

Editing Metadata - Title & filename lengths
Yeah, Windows filenames does throw a wrinkle into long titles. And I suppose also long series names, when those are included in saves/sends. It's been years since I used Windows and I simply forgot…. Now you pointed it out and I read this Naming Files, Paths, and Namespaces, (Windows Dev Center) I realize I should change how I deal with title addendums and long series names such as with hierarchical series subgroups, since we all might as well try to be interoperable as possible with filenames, for principle's sake if nothing else. For Title, off the top of my head: only append very short items such as " - 3 Ed" or " - ARC" to make title more unique for editions. And put any omnibus elements into their own empty book wishlist record, rather than appended to title. And put long title addendums in custom column Note. For Series, I need to think about it some more, and test some templates for truncating and abbreviating - - and I can probably use or adapt one of chaley's examples from calibre user manual section on Template Language. This issue is probably why he used that particular example in the first place.

Maintenance - Calibre Library Folders
Done. Made entire paragraph bold - at least until I think of some other way to emphasize it significantly but more subtly (if that's even possible….)

It may take a few days to a week before any of these other changes (and subsequent dependent changes) go into effect in the document. My general rule is that any changes in the document get tested and worked into my real-life workflow first. Though to my regret and later rework, on some occasions I screwed up by including something I hadn't tested sufficiently. Such as un-KISS series numbering for Complex Series (in Edit Metadata section), that I'd tested with insufficient number of cases when overly tired. Which I just somewhat fixed, and probably still needs more work along with the other series conventions toward being simpler. (The more specific I get, the more I get in trouble….)

travger, thanks again for the feedback.

@all, you're welcome to chime in with discussion, comment, feedback.



Edit: So what is a good compromise on title or series name length, taking into account the whole path with filename needs to fit into what, 260 char? - Would you say 40 char for title? Seems too short…. Oops, let's see. Would also need path including title + series + series_index. So series name saved or sent by template needs to be a very short abbreviation derived from longer string in Series column.

Last edited by unboggling; 09-02-2012 at 07:28 AM. Reason: minor corrections & last question
unboggling is offline   Reply With Quote
Old 09-02-2012, 10:38 AM   #11
travger
Evangelist
travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.travger ought to be getting tired of karma fortunes by now.
 
travger's Avatar
 
Posts: 469
Karma: 270594
Join Date: Aug 2010
Device: palm tx, Windows XP, Windows7
Always happy to be in the position to help; you're welcome.

You better be prepared that in Windows path starts with C:\ ...

I think it may be rather personal for what purpose anybody uses save/send and accordingly the needs for filename may be different. For example if screen is small and reader app is showing filenames, there's not much point having every filename like: Gordon Dahlquist - Miss Temple, Doctor Svenson, and ... disappearing at the edge of the screen.
That said, now I must confess that I use send/save very rarely - for me it's drag'n drop from Calibre folder to SD card.

I keep Book panel open, so Comments field is perfect for me to enter long title for the book. For example - Simple Smith: How Even a Village Fool Can Have a Fulfilling and Happy Life I enter into Comments with h5 or h6 tags, and Simple Smith into book title field.

For series I have 'Cycle' field as subseries, but if I needed to save to disk, I'd use just main series name and numeration. Cycle, Awards, Artist etc. info goes into file itself.

OMG, you speak so easily about templates and still hesitate about html ! (In Sigil you just have to click 'Insert picture'. Though I suspect you need something more complicated.)

One more point - when adding retail ebooks (or even most ePubs/mobis) it's rather safe to read metadata from the file itself. Pro side being also, that name of the downloaded file will not matter. I must add some info into custom columns manually anyway, so I'm eyeballing all the other fields too.
travger is offline   Reply With Quote
Old 09-02-2012, 11:19 AM   #12
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 16,957
Karma: 10252998
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Astak Pocket Pro, K4NT,Galaxy Tab 2
Quote:
Originally Posted by travger View Post
You better be prepared that in Windows path starts with C:\ ...
Only absolutely true if your system has a Single HD (with a single Windows accessible) partition)
Modern Systems can have multiple R/W drives (you should not have to worry about path length with Read Only media ), some removable. some networked. On a few occasions, I have had more than half of the available drive letters in use (data collection and processing. Hoo-Ray for the old Netware 4.x. No problems.)
theducks is offline   Reply With Quote
Old 09-03-2012, 04:24 PM   #13
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Revision notes:

Approach:
Added brief intro to clarify what it is and isn't.

Adding Books - Choose Method for Adding Books:
Added sentence that reading from file contents usually works well for retail-quality books from good sources.

Editing Metadata, Title or Series - and path/filename length issue:
Shortened the appends to Title. Mentioned an alternative for people wanting shorter titles might be putting that metadata into comments or custom column(s). Added separate paragraph (after Authors) about string length in Series and Title. Decided not to go into specifics about calibre automatically truncating when necessary to deal with various file system path/filename limitations, or the tech specs of those limitations (that would have been off-topic; and I don't know all the details). But added a few links under Filename Limitations in Links section. And added links for template info and examples in separate paragraph Set Templates in Reading Books section, and added links in Sending Books in Preferences section.

Other issues:
Still thinking about the issues with bulk convert, delete original format, and preferred fix with Sigil. Any changes there may be awhile since I want to get comfortable with any changes in real-life workflow before revising the document.

But I'm tending toward never deleting original format. In most cases that would give every book 2 formats, Preferred and Original. In the case of Preferred to Preferred, it would be Preferred and Original_Preferred, requiring Tweak save_original_format=true. For users learning how to fix books it's probably more important to keep the original formats, than for experienced book fixers who know what they're doing. I'm still in the learning how category, particularly with Sigil.

And I finally learned how to anchor a selected graphic to a paragraph in OpenOffice, preceding add of ODT into calibre. With menu command Format/Anchor/To Paragraph. All I had to do was look.

I'm also tending toward always bulk converting after adding books, for all books. For me using EPUB as preferred format, that makes sense for the assessment in calibre viewer. With further assessment of original format if necessary in other reader app for that format.

Last edited by unboggling; 09-04-2012 at 05:45 AM. Reason: revision of revision notes :)
unboggling is offline   Reply With Quote
Old 09-04-2012, 12:33 AM   #14
mraxehandle
Junior Member
mraxehandle began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Feb 2007
Device: Palm T3
I still don't understand why so many people use forums as a kind of wiki (I think of XDA in particular.) It seems strange to me, given how many great wiki-type tools are out there.Does MobileRead or Calibre have a wiki space that can be used?
mraxehandle is offline   Reply With Quote
Old 09-04-2012, 02:10 AM   #15
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,058
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad2, Kindle3KB
Quote:
Originally Posted by mraxehandle View Post
I still don't understand why so many people use forums as a kind of wiki (I think of XDA in particular.) It seems strange to me, given how many great wiki-type tools are out there.Does MobileRead or Calibre have a wiki space that can be used?
Welcome to MobileRead Forums.

MobileRead Wiki Main Page, links to ebooks, libraries, stores, and information on devices, readers, tools, and formats.

There is a calibre article on MobileRead wiki. AFAIK nothing prevents there being other calibre wiki pages with different focus or content.

These are some reasons this document is here:
  1. It is still under development, being added to and refined, with rapid revisions.
  2. Feedback spurs that along.
  3. I want to keep control of the edits. AFAIK, any registered user can edit a wiki article. I might be wrong because I never contributed to a wiki before, and don't know specifics of that process.
  4. The focus, as currently written and structured, is on how "I" manage ebooks with calibre, my needs, strategy, workflow, configs, and settings and how they interrelate. Not necessarily on how other people manage ebooks with calibre, which is usually different for each person. (General discussion of what other users do is mostly for context or contrast. Any discussion of methods to choose from is usually for context or contrast, and/or workflow contingencies.)
  5. I suspect most of the people reading this in a calibre forum are more calibre-focused than a broader group reading it as a wiki article.

If I'm wrong about anything in that reasoning, anybody, please let me know.

Last edited by unboggling; 09-04-2012 at 06:36 AM. Reason: clarify
unboggling is offline   Reply With Quote
Reply

Tags
calibre workflow, ebook management strategy, ebook management workflow

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
PRS-T1 Manage Collections in Calibre (Help!) FatCat0 Sony Reader 19 08-11-2012 12:00 PM
How to find & manage ebooks from various apps? rapidlanguage Library Management 3 01-06-2012 08:13 AM
Development Using Calibre to manage eDGe library mrspaceman enTourage Archive 76 05-12-2011 12:38 PM
Neo How to manage ebooks? ivanm BeBook 11 08-19-2010 11:01 AM
How do you manage your read queue with ebooks? DuncanWatson General Discussions 7 05-14-2010 01:30 PM


All times are GMT -4. The time now is 10:36 PM.


MobileRead.com is a privately owned, operated and funded community.