Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Conversion

Notices

Reply
 
Thread Tools Search this Thread
Old 11-19-2011, 11:02 AM   #1
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 320
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Kindle Fire periodicals need EXTH[501]=NWPR

Kindle Fire uses the EXTH[501] field to decide if the newsreader should be used for a document. At present calibre doesn't set that for recipe output at all, so recipe output is viewed by Kindle Fire as a book. To get the (much nicer) newsreader calibre has to set EXTH[501]='NWPR'. This will also cause the document to be placed in the "Newsstand" tab on the home page of the Fire.

There are two places where this should be set--in mobi/writer.py and mobi/writer2/main.py. (The code seems to have been reorganized since I last worked on it so I'm unclear as to whether the tweak new_mobi_writer is still relevant.)

In mobi/writer.py, replace lines 1578-1583 with
Code:
        # Write cdetype
        if not self.opts.mobi_periodical:
            data = 'EBOK'
            exth.write(pack('>II', 501, len(data)+8))
            exth.write(data)
            nrecs += 1
        else: # fix for Kindle Fire--use newsreader for this
            data = 'NWPR'
            exth.write(pack('>II', 501, len(data)+8))
            exth.write(data)
            nrecs += 1
In mobi/writer2/main.py, replace lines 537-547 with
Code:
        # Write cdetype
        if not self.is_periodical and not self.opts.share_not_sync:
            exth.write(pack(b'>II', 501, 12))
            exth.write(b'EBOK')
            nrecs += 1
        elif self.is_periodical:
            # fix for Kindle Fire--use newsreader for this
            exth.write(pack(b'>II', 501, 12))
            exth.write(b'NWPR')
The Fire newsreader is obviously derived from the Kindle Android newsreader, but it's interesting to note the latter doesn't require the EXTH[501] to be set. It looks like Kindle Fire is using the EXTH[501] to categorize documents, and then decides how to display them based on the category rather than looking at the MOBI type.
nickredding is offline   Reply With Quote
Old 11-19-2011, 05:32 PM   #2
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 320
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Update: this works as expected when transferring files to the Fire via USB but for some reason emailing the files doesn't have the same result. In that case the files still show up vanilla docs. I will investigate and report back.
nickredding is offline   Reply With Quote
 
Enthusiast
Old 11-19-2011, 09:33 PM   #3
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
writer.py is now no longer used, unless asked for explicitly by a tweak. By default calibre uses writer2.py. There is another thread on this forum which has some info about news downloads needing to have a .pobi extension to be recognized.
kovidgoyal is online now   Reply With Quote
Old 11-19-2011, 09:35 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Also, I have seen values NWPR and MAGZ for that field (corresponding to doctype of 0x101 and 0x103.
kovidgoyal is online now   Reply With Quote
Old 11-19-2011, 09:43 PM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Committed code to set 501 for periodicals.
kovidgoyal is online now   Reply With Quote
Old 11-20-2011, 11:13 AM   #6
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 320
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
E-mailing periodicals is problematic

Amazon steps on any EXTH[501] value in a document being emailed to a Kindle and replaces it with "PDOC". Therefore, you'll only get the newsreader/newsstand treatment if you copy the files to the Fire manually.

Amazon has rendered the calibre email delivery option useless (unless you are willing to put up with navigating your periodicals as an ebook). I can't think of any way of fixing this short of developing an app that modifies the files after they are delivered, and even that would involve manually invoking the app for each periodical file.

I'm pretty sure Amazon won't be sympathetic since it doesn't affect Amazon content, so I'm not going to bother complaining in their forums (they typically ignore complaints that involve non-Amazon content).
nickredding is offline   Reply With Quote
Old 11-20-2011, 11:59 AM   #7
wallcraft
reader
wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.
 
wallcraft's Avatar
 
Posts: 6,979
Karma: 5183568
Join Date: Mar 2006
Location: Mississippi, USA
Device: Kindle 3 and Fire
Quote:
Originally Posted by nickredding View Post
I can't think of any way of fixing this short of developing an app that modifies the files after they are delivered, and even that would involve manually invoking the app for each periodical file.
There might be room for an app that edits the EXTH[501] value for files on the device and triggers the local sync that is currently done by connecting and disconnecting USB to a computer. Even folks who did not care about EXTH might like the local sync as an alternative to a reboot. The app developer would have to work out how to do the on-device sync though.
wallcraft is offline   Reply With Quote
Old 11-20-2011, 12:21 PM   #8
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 9,311
Karma: 42858084
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Amazon has rendered the calibre email delivery option useless (unless you are willing to put up with navigating your periodicals as an ebook).
Hasn't emailing a document to a Kindle device always resulted in EXTH[501] being set to PDOC, though? That's why emailing always used to break the different synchronization kludges in the past.

Nevermind: I see what you're saying. In the past, EXTH[501] wasn't used to differentiate between ebook or periodical. So it didn't matter (in that regard) that it was being set to PDOC... the reader app on the Kindle still knew it was a periodical.

Last edited by DiapDealer; 11-20-2011 at 12:27 PM.
DiapDealer is offline   Reply With Quote
Old 11-20-2011, 12:34 PM   #9
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 320
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Quote:
Originally Posted by kovidgoyal View Post
Committed code to set 501 for periodicals.
There's still a problem with the fix. Calibre-generated periodicals all have the MOBI type set to 0x103 which results in EXTH[501]=MAGZ instead of NWPR. This causes the Fire to display the periodical TOC as a linear list of articles without a drop-down menu to jump to a specific section. I suspect the MAGZ classification is intended for periodicals that have only one section. For multi-section periodicals we really need EXTH[501]=NWPR.

I looked at the code and I see
Code:
        if oeb.metadata.publication_type:
            x = unicode(oeb.metadata.publication_type[0]).split(':')
            if len(x) > 1:
                pt = x[1].lower()
        self.publication_type = pt
and then the MOBI type is set as
Code:
{'newspaper':0x101}.get(self.publication_type, 0x103)
so obviously the oeb.metadata.publication_type isn't being set to 'newspaper' since the type always ends up as 0x103.

I don't see any preferences that would affect oeb.metadata.publication_type, so I suspect there needs to be a change to ensure multi-section periodicals get the type 0x101, or make all periodicals (0x101 and 0x103) have EXTH[501]=NWPR.
nickredding is offline   Reply With Quote
Old 11-20-2011, 12:37 PM   #10
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 320
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Quote:
Originally Posted by nickredding View Post
Amazon has rendered the calibre email delivery option useless (unless you are willing to put up with navigating your periodicals as an ebook).
I should have qualified this by saying this only applies to the Fire (and possibly the KTouch which mnay have the same behaviour of using EXTH[501] to decide what reader to use).
nickredding is offline   Reply With Quote
Old 11-20-2011, 09:19 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
OK, committed code to set it to 0x103 and MAGZ only if the periodical is flat.
kovidgoyal is online now   Reply With Quote
Old 11-21-2011, 01:15 AM   #12
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 320
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Thanks for the quick fixes. If only Amazon was as responsive!
nickredding is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
News / periodicals date on the kindle prophet Calibre 3 12-04-2010 06:05 PM
Issue 501: images with same name in different directories collide in final epub file Ray Tayek Sigil 32 11-27-2010 01:46 AM
Amazon to enact 70 percent revenue share for Kindle periodicals carld News 8 11-09-2010 09:35 AM
Possible to treat periodicals as...periodicals? Spankypoo Calibre 1 01-23-2010 10:06 PM
exth type, exth data AlexBell Reading and Management 4 07-06-2008 11:51 PM


All times are GMT -4. The time now is 11:13 AM.


MobileRead.com is a privately owned, operated and funded community.