V5.0.0 alpha channel release

[nqk]

Wireless connection crashes on my Nexus 7, while there's no issue with other devices. Here is the debug
https://app.box.com/s/nahnf27unudt5z07fbx8e15d47pj7i3w

[chaley]

Originally Posted by nqk

Wireless connection crashes on my Nexus 7, while there's no issue with other devices. Here is the debug
https://app.box.com/s/nahnf27unudt5z07fbx8e15d47pj7i3w

You must have uninstalled and reinstalled CC, without restoring from backup.

Google changed a policy in Lollipop that causes CC to crash on the first wireless connection after install. The alpha versions see this new policy because of the compatibility lib.

Fixed in next release.

[chaley]

V5.0.0.6 has been released.

Change log:

• Start work to support the new calibre content server. This server is not generally available. My main concern at the moment is that I don't break anything when using the current calibre server.
• [BUG FIX]Fix crash on a first-time wireless device connection (first connect after installation) on devices running Lollipop and later and with Google Play installed.

[nqk]

Originally Posted by chaley

You must have uninstalled and reinstalled CC

Actually no, I didn't uninstall it. However it's now ok with the new version.

Regarding the new content server, are there anything to be excited about?

[chaley]

Originally Posted by nqk

Regarding the new content server, are there anything to be excited about?

From CC's point of view the major new feature is the ability to switch libraries, similar to what one can do in the cloud connection.

The content server UI will eventually support modification of metadata, and perhaps more. CC won't play a part in this but will "see" the results. For example, you could use the content server to change some tags, then use CC to re-download the book so it has the new tags.

[nqk]

Oh, in other words, CC will still be on the read-only mode and, since I'm relying on Cloud library, I will have to keep using Google form (then save as cvs, then import to Calibre when I'm using it) to update read status, comments, etc.

Ah, dear Chaley, I admit I quit using the yes/no column for read status, such binary option is not enough for me. I'm using now a text column with 4 preset values (read, reading, abandoned, planned). I lose the green check in CC but it's easier to keep track of my reading.

One more thing, I've changed system language to Vietnamese and choose "system" in CC language ==> the language interface in CC is stable.

[chaley]

Originally Posted by nqk

One more thing, I've changed system language to Vietnamese and choose "system" in CC language ==> the language interface in CC is stable.

This means that my changes to help stabilize setting the language didn't fix it? I have added another "fix" that will be in the next alpha release.

I wish I could see this happen on one of my devices.

[chaley]

V5.0.0.7

Change log:

• Support for the new (unreleased) calibre content server. There should be no change in functionality when using the existing content server.
• Restore the "busy spinner" in the content server connection.
• [BUG FIX]Fix rare crash when adding books with strange internal metadata.
• [BUG FIX]Yet another try to make a selected language "stick".

I think this will be the last alpha release in this cycle.

@Translators: I am pushing new and changed strings to Onesky

[jackie_w]

Finally my Internet service has been fixed!

Re: 5.0.0.7

I've been testing transfers
a - via WiFi
b - via Content Server (both short-press and long-press)

with Settings - Filenames set to
1 - 'Use calibre filenames'
2 - 'Use CC template everywhere'/'Replace existing book files'

I set slightly different templates in the SmartDevice App driver and CC template so I would see filename differences.

Transferred filenames were always what I would have expected them to be and resending the same book resulted in files being correctly renamed when necessary.

However, I did see some unexpected behaviour when using the Content Server to try to resend a book.

• With Show dialog on book tap set:
  - long-press did not do an immediate transfer, it brought up the same pop-up (Download or Show Metadata) as when doing a short-press.
  - whichever type of press, selecting Download from this pop-up did not resend the book.
  
• With Always show metadata set:
  - short-press opened Book Details and pressing the DOWNLOAD button (as expected) gave me an 'Are you sure you want to overwrite?' pop-up which did then do the resend.
  - long-press did not do an immediate transfer, it did exactly the same as short-press. I'm not sure whether this is intended behaviour or not.

[chaley]

Originally Posted by jackie_w

Finally my Internet service has been fixed!

I set slightly different templates in the SmartDevice App driver and CC template so I would see filename differences.

Transferred filenames were always what I would have expected them to be and resending the same book resulted in files being correctly renamed when necessary.

Thanks for verifying and letting me know.

However, I did see some unexpected behaviour when using the Content Server to try to resend a book.

• With Show dialog on book tap set:
  - long-press did not do an immediate transfer, it brought up the same pop-up (Download or Show Metadata) as when doing a short-press.
  - whichever type of press, selecting Download from this pop-up did not resend the book.
  
• With Always show metadata set:
  - short-press opened Book Details and pressing the DOWNLOAD button (as expected) gave me an 'Are you sure you want to overwrite?' pop-up which did then do the resend.
  - long-press did not do an immediate transfer, it did exactly the same as short-press. I'm not sure whether this is intended behaviour or not.

Both of these are intentional and come from the same decision about how the download queue works.

The first rule: the book downloaded by queuing it (via long press or "Download all") downloads the preferred format. The second rule: books already on the device are ignored and not resent.

Thus a long-press on a book already on the device is ignored. Better said, rather than do nothing I turn it into a short press where the user can explicitly choose to download the book and also choose the format. I suppose that it is an error to show the "choose" dialog with the download button enabled because, as you note, the download button doesn't do anything. Consistency argues that the dialog should show. Usability perhaps argues that it should go directly to book details, or perhaps popup a "checkable help" dialog telling you why the download isn't happening.

Some people have said that a long-press should be happy to silently re-download a book. I am uncomfortable with that, because changing the book is "destructive" and can change the behavior of a reader app. I am also uncomfortable with popping up the "Are you sure" dialog, but less so. "Download all" doesn't provide an opportunity for a dialog.

[chaley]

Originally Posted by jackie_w

ETA: Mea maxima culpa. I really need to remember that when alpha/beta-testing I must revert to using the official SmartDevice App Interface 0.0.1 driver, not the subclassed personal one I've been using for ages to force a small CSS change during book transfer. I've redone the Content Server tests (long-press and short-press) for 2 not-on-device-books. Filenaming is working correctly in 5.0.0.4 for the 'Use calibre's file names' setting.

I realise it is not reasonable for me to pursue why my personalised driver fails to rename correctly at this time. Maybe later (much later) when you have some free time after the v5 production release.

Sorry, I didn't notice this edit.

Depending on when you subclassed the driver and depending on what methods you overloaded, you might be missing some of the CC/calibre "feature negotiation". The two programs converse about what each is capable of doing and pick the common subset.

In particular I suspect that the canSupportLpathChanges options introduced somewhere around calibre 2.30 might be missing in your version, especially if you overloaded _put_file.

[jackie_w]

Originally Posted by chaley

Sorry, I didn't notice this edit.

Depending on when you subclassed the driver and depending on what methods you overloaded, you might be missing some of the CC/calibre "feature negotiation". The two programs converse about what each is capable of doing and pick the common subset.

In particular I suspect that the canSupportLpathChanges options introduced somewhere around calibre 2.30 might be missing in your version, especially if you overloaded _put_file.

I'm tempted to say 'forget about it'. For personal consumption (I only ever use WiFi to transfer) my hack seems to work just as well as it ever has. I've chosen Settings - Filenames - Use CC template everywhere/Replace existing book files as my new standard setting.

I just retested with v5.0.1.1 beta and calibre v2.51 (32bit). All is well with file- naming & renaming when using CC-WiFi with hacked driver.

If you want to know more - and I won't blame you if you don't - I mean, you could be doing useful stuff for thousands of paying customers who aren't trying to be a bit too clever for their own good ...

As far as I can tell the only smart_device_app method affected would be upload_books. But even there, my hack only does 2 minor edits to an epub's CSS files (using the official calibre container) before handing back to the official smart_device_app upload_books method.

This is the whole of the hacked driver, it's very small. Its purpose is to allow fully working custom fonts in epubs, without embedding. Many Android reader apps claim to do this, unfortunately most of them make a holy mess of it (IMO). At least with the hack I can get Mantano and Pocketbook to display reg/ital/bold/boldital correctly without sacrificing sans-serif/monospace along the way.

Spoiler:

import os
from calibre.devices.smart_device_app.driver import SMART_DEVICE_APP
from calibre.devices.usbms.driver import debug_print
from calibre.ebooks.oeb.polish.container import get_container

EPUB_EXT = '.epub'
SMART_DEVICE_APP_IMPRULE = '@import url(res:///../mantano.css);'
SMART_DEVICE_APP_ADDRULE = 'body {font-family: serif}'

class SMART_DEVICE_APP_CSSADD(SMART_DEVICE_APP):
    ''' Tweaked driver for Calibre Companion Android device.
    This driver automatically modifies ePub files by inserting a 
    CSS import rule as 1st rule of each css file. E.g.
        @import url(res:///../external_sd/ebooks/mantano.css);
    Then all files are transferred as normal '''

    name = 'Smart_Device_App_CSSAdd'
    author = 'jackie_w'
    description = 'Communicate with Calibre Companion Android device. Based on the existing SMART_DEVICE_APP driver by chaley. Inserts CSS import rule to epub existing CSS file(s).'

    minimum_calibre_version = (0, 9, 29)
    version = (0, 0, 3)

    def initialize(self):
        super(SMART_DEVICE_APP_CSSADD, self).initialize()

    def _modify_epub(self, file, metadata):
        debug_print("Smart_Device_App_CSSAdd:_modify_epub:Processing {0} - {1}".format(metadata.author_sort, metadata.title))
        container = get_container(file)
        cssnames = [n for n in container.name_path_map if n.endswith('.css')]
        for cssname in cssnames:
            newsheet = container.parsed(cssname)
            oldrules = len(newsheet.cssRules)
            # insert @import rule as first rule
            newsheet.insertRule(SMART_DEVICE_APP_IMPRULE, 0)
            # append constant rule as last rule
            newsheet.insertRule(SMART_DEVICE_APP_ADDRULE, len(newsheet.cssRules))
            debug_print("Smart_Device_App_CSSAdd:_modify_epub:CSS rules {0} -> {1} ({2})".format(oldrules, len(newsheet.cssRules), cssname))
            container.dirty(cssname)
        container.commit()    
        return True

    def upload_books(self, files, names, on_card = None, end_session = True, metadata = None):
        new_files = []
        new_names = []
        new_metadata = []
        i = 0
        for file, n, mi in zip(files, names, metadata):
            self.report_progress(i / float(len(files)), "Processing book: {0} by {1}".format(mi.title, " and ".join(mi.authors)))
            ext = file[file.rfind('.'):]
            if ext == EPUB_EXT:
                self._modify_epub(file, mi)
                new_files.append(file)
                new_names.append(n)
                new_metadata.append(mi)
            else:
                new_files.append(file)
                new_names.append(n)
                new_metadata.append(mi)
            i += 1

        self.report_progress(0, 'Working...')
        result = super(SMART_DEVICE_APP_CSSADD, self).upload_books(new_files, new_names, on_card, end_session, new_metadata)
        return result

[chaley]

Originally Posted by jackie_w

I'm tempted to say 'forget about it'. For personal consumption (I only ever use WiFi to transfer) my hack seems to work just as well as it ever has. I've chosen Settings - Filenames - Use CC template everywhere/Replace existing book files as my new standard setting.

I think I am going to take you up on the "forget about it" offer.

I glanced through your code and the underlying driver and don't see any reason it should misbehave. This is one instance in the driver where __class__.__name__ is used but only to name a file, so that should be OK. I am not sure why you are creating new arrays in your version of upload_books, given that the underlying data structures are not being modified, but I also don't see a reason why this would affect anything.

YAMOL (Yet Another Mystery Of Life)

[jackie_w]

Originally Posted by chaley

I am not sure why you are creating new arrays in your version of upload_books, given that the underlying data structures are not being modified, but I also don't see a reason why this would affect anything.

Easy to answer. Most of the code was stolen inspired by someone else's modified driver code ... and as it seemed to work ... No-one told me I was supposed to understand it, as well Where's the fun in that?

[Terisa de morgan]

Originally Posted by jackie_w

Easy to answer. Most of the code was stolen inspired by someone else's modified driver code ... and as it seemed to work ... No-one told me I was supposed to understand it, as well Where's the fun in that?

Looks like my daily work