Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > More E-Book Readers > iRex > iRex Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 07-09-2011, 04:16 PM   #76
Iņigo
Guru
Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.
 
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
Lightbulb metadata: proof of concept

Hi,

I've been very productive this week developing a DR global.db metadata management tool.

DESCRIPTION:
What is this? an evolution of my old "icons" program, a more ambicious command line application to add/update/delete metadata and thumbnails of books. Or if you prefer, an in-progress mdbindex replacement.

Purpose? add thumbnails or other metadata quickly, as mdbindex crashes or is too slow.

What formats does it support? .desktop, .pdf, .fb2, .epub. Take a look at Issues & Bugs section below.


USE:
It can be used from the DR command-line (rxvt or usbnet) or from your own computer.
Note the binary in the package attached is only for the DR, in case you want to run this program on your computer you must compile it before. If anyone wants I can attach a linux x86_64 binary.

As most of the options are self-explaining I will only mention some of them.

Case 1:
Let's suppose you have mounted the DR with the USB cable in /media/DR800 and you have your PC-compiled version of the program in /home/inigo/tmp, which is your current working directory.

Then you type:

Code:
$ cd /home/inigo/tmp
$ ./metadata -fs -v -l --sdcard=/media/DR800  --action=report   /media/DR800/Books/Fiction  /media/DR800/Books/NonFiction
-fs: as I'm using DR800+ my thumbnails are stored in file system, not in global.db
-v: show some information on screen while running
-l: generate log file in current directory
--sdcard=/media/DR800 path where to find SD card 'global.db' file and 'System' folder
--action=report: create a report of the files
directories with books

This commands is not fully correct, but we will explain after next case.


Case 2:
You have copied a folder with books from the DR to the current folder of your PC for faster processing. You have also copied global.db (and System/.covers in case of DR800+) to your current folder.

Code:
$ cd /home/inigo/tmp
$ ./metadata -v -l --type=pdf,epub  --add-prefix=/media/mmcblk0p1/Books  --action=report   Fiction
--type=pdf,epub only process PDF and EPUB files (not FB2 or DESKTOP)
--add-prefix=/media/mmcblk0p1/Books

No --sdcard or --del-prefix as they default to current directory
With the "--add-prefix=/media/mmcblk0p1/Books" we were modifying filenames path, so "/home/inigo/tmp/Fiction/book1.pdf" will be converted to "/media/mmcblk0p1/Books/Fiction/book1.pdf", as we drop "/home/inigo/tmp" (--del-prefix defaults to current folder, remember) and then add "/media/mmcblk0p1/Books"
Then we can run any other action and copy global.db (and System/.covers) back to the DR.


Back to Case 1: when SD card is inserted into the DR, it is mounted in /media/mmcblk0p1 not in /media/DR800 as in the computer, so the correct command is:
Code:
$ cd /home/inigo/tmp
$ ./metadata -fs -v -l --sdcard=/media/DR800  --del-prefix=/media/DR800  -add-prefix=/media/mmcblk0p1  --action=report   /media/DR800/Books/Fiction  /media/DR800/Books/NonFiction



Important:
- There are lots of options, but usually you won't need to specify most of them, only action and directory
- Remember to backup your global.db before running this program
- Also note that depending on the number of files it can take some time to finish


Code:
Manage books metadata for iRex DR series eReaders.
Copyright (C) 2011  Iņigo Serna <inigoserna@gmail.com>
Released under GPL v3+

Usage: metadata  [options] directory1 [directory2] [...]

    -h, --help            Display this help
    -v, --verbose         Show additional information on screen
    -l, --log             Write a log in file metadata-{timestamp}.log

    -fs, --thumbs-in-fs   Whether thumbs are stored on file system (default false, in global.db)
                          This option is needed if you are using DR800+ version 4 or higher

    --sdcard=/path/to     Folder where to find SD card 'global.db' file and 'System' folder (default current directory, then /media/mmcblk0p1)
    --del-prefix=/path    Prefix path to be removed from the start of book path in database (default current directory)
    --add-prefix=/path    Prefix path to be added to the start of book path in database (default current directory)

    --type=TYPES          File types to analyze (default all: desktop, pdf, fb2, epub)
                          all
                          item1,item2,item3 (no spaces allowed between items)
                          item

    --action=ACTION       Action to perform (default report)
                          report: show book information
                          add: add new books to database
                          add-metadata: add new books (without thumbnails) to database
                          add-thumbs: add missing thumbs to database
                          overwrite: force update books metadata to database
                          overwrite-metadata: force update books metadata (no thumbnails) to database
                          overwrite-thumbs: force update books thumbs to database
                          update: add if new book or update
                          update-metadata: add if new book or update metadata
                          update-thumbs: add if new book or update thumbs
                          delete: remove entries from database
                          delete-thumbs: remove thumbs from database
                          vacuum: clean database

ISSUES & BUGS:
- overwrite-metadata: update any other field? now only author and title
- does sort by last added work with books added with this program?
- .pdf with first page only contains text -> thumbnail will look blank
- .epub and .fb2 books without images -> no thumbnail will be generated
- .epub with error message "unzip: inflate error": unzip program in DR is old and can't uncompress the file
- .fb2 books not saved with UTF8 encoding fail


SOME IDEAS FOR THE FUTURE:
- a GUI tool
- allow a different source to specify book metadata (title, author)
- add Mackx's "no-update" tag and many tags features to DR800+
- handle more than one tag here
- replace mdbindex?
. we should manage every registered file and dirs
. delete entries and dirs present in db but not in file system


EDIT 2011/07/11 09:00 GMT: slightly modified package

Iņigo
Attached Files
File Type: gz metadata.tar.gz (51.7 KB, 572 views)

Last edited by Iņigo; 07-11-2011 at 04:57 AM.
Iņigo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Mobiperl Mobiperl - Perl tools for handling MobiPocket files tompe Kindle Formats 890 01-17-2021 06:16 PM
Large number of books on memory card pwalker8 Sony Reader 8 03-24-2009 02:20 PM
huge pdf files on iliad yolle iRex 7 08-23-2008 10:59 AM
Fictionwise Handling Fee Against Credit Card Terms of Service? Gideon News 9 08-15-2008 02:13 PM
Huge RTF files after conversion TheMadBrewer Sony Reader 2 01-01-2007 02:55 AM


All times are GMT -4. The time now is 11:12 AM.


MobileRead.com is a privately owned, operated and funded community.