Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 11-27-2011, 06:26 PM   #1
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 324
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
MOBI periodical index thumbnail image

The MOBI format has the ability to associate an image with an index (TOC) entry which, if present, will be scaled and displayed by Kindle in the TOC list view. Amazon-generated periodicals make use of this and I would like to extend recipe functionality to allow recipes to optionally add an image reference to articles.

The easiest way to do this would be to use populate_article_metadata to scan the processed article, extract the url of an image to reference in the index (if there is such an image), and store that somewhere where MobiWriter can retrieve it. Any other output writer could make use of this or (by default) ignore it by not accessing the references.

I'm not familiar enough with the transition from input to conversion to output to be able to see where the references could be stashed for retrieval by an output writer. Is the OEB representation all on disk between input and output, or is there an in-memory representation of the article metadata that could be extended to include index image references? Can someone point me in the right direction?
nickredding is offline   Reply With Quote
Old 11-27-2011, 10:44 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
That will require rather a lot of changes to various parts of calibre. To start with you would have to add a image_href attribute to the Article class and populate it with the value of the src attribute of the correct img tag in populate_article_metadata (you would have to make the link relative the the top level directory of the oeb book). Then change metadata/toc.py to support image_href, then change the TOC class in oeb/base.py then the _toc_from_navpoint() method in oeb/reader.py and finally MobiWriter.
kovidgoyal is offline   Reply With Quote
Old 11-27-2011, 11:11 PM   #3
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 324
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Kovid - Thanks for the advice--seems pretty straighforward. Can I assume you'll merge the changes if I develop and test them out?
nickredding is offline   Reply With Quote
Old 11-27-2011, 11:33 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Yeah, though given that it involves path manipulations it would need to be tested in both windows and a POSIX platform.
kovidgoyal is offline   Reply With Quote
Old 12-04-2011, 02:49 PM   #5
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 324
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Kovid - I have tested the changes on both Windows and Linux (Ubuntu). I don't have any way of testing on OS-X but I'm guessing the changes will work on that as well.

I have attached the modified files (from the latest release) as well as modified NY Times recipes to use the thumbnail image capability.

Folowing are the changes required (line numbers refer to the current files before any changes are applied).

Let me know if you see any issues.


calibre/web/feeds/__init__.py

Insert after line 33:
Code:
        self.toc_thumbnail = None
Insert after line 89:
Code:
TOC thumb   : %s
Replace line 92:
Code:
'''%(self.title, self.url, self.author, self.summary[:20]+'...', ('None' if self.toc_thumbnail is None else self.toc_thumbnail), self.localtime.strftime('%a, %d %b, %Y %H:%M'),
calibre/web/feeds/news.py

Insert before line 656:
Code:
    def add_toc_thumbnail(self, article, src):
        '''
        Call this from populate_article_metadata with the src of an img tag you want to
        appear as a thumbnail image in the articles list view of the table of contents
        (Kindle feature)
        '''
        article.toc_thumbnail = re.sub(r'^.*feed','feed',src.replace('\\','/'),flags=re.IGNORECASE)
Insert after line 1287:
Code:
                    idx = a.toc_thumbnail
                    if not idx:
                        idx = None
Replace lines 1293-1294:
Code:
                    parent.add_item('%sindex.html'%adir, None, a.title if a.title else _('Untitled Article'),
                                    play_order=po, author=auth, description=desc, toc_thumbnail=idx)
calibre/ebooks/metadata/toc.py

Replace lines 32-34:
Code:
    def __init__(self, href=None, fragment=None, text=None, parent=None, play_order=0,
                 base_path=os.getcwd(), type='unknown', author=None,
                 description=None, toc_thumbnail=None):
Insert after line 45:
Code:
        self.toc_thumbnail = toc_thumbnail
Replace lines 74-80:
Code:
    def add_item(self, href, fragment, text, play_order=None, type='unknown',
            author=None, description=None, toc_thumbnail=None):
        if play_order is None:
            play_order = (self[-1].play_order if len(self) else self.play_order) + 1
        self.append(TOC(href=href, fragment=fragment, text=text, parent=self,
                        base_path=self.base_path, play_order=play_order,
                        type=type, author=author, description=description, toc_thumbnail=toc_thumbnail))
Insert after line 271:
Code:
            idx = getattr(np, 'toc_thumbnail', None)
            if idx:
                elem.append(C.meta(idx, name='toc_thumbnail'))
calibre/ebooks/oeb/base.py

Replace lines 1619-1651 with:
Code:
class TOC(object):
    """Represents a hierarchical table of contents or navigation tree for
    accessing arbitrary semantic sections within an OEB data model book.

    Acts as a node within the navigation tree.  Provides list-like access to
    sub-nodes.  Provides the follow node instance data attributes:

    :attr:`title`: The title of this navigation node.
    :attr:`href`: Book-internal URL referenced by this node.
    :attr:`klass`: Optional semantic class referenced by this node.
    :attr:`id`: Option unique identifier for this node.
    :attr:`author`: Optional author attribution for periodicals <mbp:>
    :attr:`description`: Optional description attribute for periodicals <mbp:>
    :attr:`toc_thumbnail`: Optional toc thumbnail image
    """
    def __init__(self, title=None, href=None, klass=None, id=None,
            play_order=None, author=None, description=None, toc_thumbnail=None):
        self.title = title
        self.href = urlnormalize(href) if href else href
        self.klass = klass
        self.id = id
        self.nodes = []
        self.play_order = 0
        if play_order is None:
            play_order = self.next_play_order()
        self.play_order = play_order
        self.author = author
        self.description = description
        self.toc_thumbnail = toc_thumbnail

    def add(self, title, href, klass=None, id=None, play_order=0, author=None, description=None, toc_thumbnail=None):
        """Create and return a new sub-node of this node."""
        node = TOC(title, href, klass, id, play_order, author, description, toc_thumbnail)
        self.nodes.append(node)
        return node
calibre/ebooks/oeb/reader.py

Replace lines 374-375:
Code:
            indeximageElement = xpath(child,
                    'descendant::calibre:meta[@name = "toc_thumbnail"]')
            if indeximageElement :
                toc_thumbnail = indeximageElement[0].text
            else :
                toc_thumbnail = None

            node = toc.add(title, href, id=id, klass=klass,
                    play_order=po, description=description, author=author,
                           toc_thumbnail=toc_thumbnail)

calibre/ebooks/mobi/writer2/indexer.py

Replace line 139 with:
Code:
            'author_offset': 71,
(Note that the original value of 73 for author_offset is incorrect and this change doesn't relate specifically to the TOC thumbnail image changes.)

Insert after line 756:
Code:
                if art.toc_thumbnail is not None:
                    if art.toc_thumbnail in self.serializer.images:
                        article.image_index = self.serializer.images[art.toc_thumbnail]
Attached Files
File Type: zip nytimes.zip (16.7 KB, 326 views)
File Type: zip python files.zip (56.5 KB, 333 views)
nickredding is offline   Reply With Quote
Old 12-04-2011, 07:20 PM   #6
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 324
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Kovid - sorry I omitted this from the previous message (it is in the file attachment however).

In calibre/ebooks/mobi/writer2/indexer.py replace line 228 with
Code:
                    buf.write(encint(val-1 if et==0x3F and attr=='image_index' else val))<\
nickredding is offline   Reply With Quote
Old 12-04-2011, 10:26 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,778
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Can you open a bug report for this, I am rather busy this week, so it may be a little while before I can get to reviewing this.
kovidgoyal is offline   Reply With Quote
Old 12-04-2011, 11:04 PM   #8
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 324
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Quote:
Originally Posted by kovidgoyal View Post
Can you open a bug report for this, I am rather busy this week, so it may be a little while before I can get to reviewing this.
Ok will do
nickredding is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Thumbnail Image Distorted and Font Size Sigil Jay22 Sigil 2 11-03-2011 03:09 PM
Index: Making a linked index in epub virtual_ink ePub 21 10-19-2011 11:23 PM
Thumbnail for cover image has white borders Mookiemon ePub 0 07-06-2011 12:40 PM
HTML to Mobi conversion with very wide tables and Index tab sjahangi Conversion 0 04-21-2011 02:30 PM
mobi catalog = periodical, ePub catalog = book trekchick Library Management 7 03-12-2011 01:11 PM


All times are GMT -4. The time now is 04:40 AM.


MobileRead.com is a privately owned, operated and funded community.